arm64: provide pseudo NMI with GICv3
Subsystems (7)
cover arm64: provide pseudo NMI with GICv3
Hi,
This series is a continuation of the work started by Daniel [1]. The goal is to use GICv3 interrupt priorities to simulate an NMI.
To achieve this, set two priorities, one for standard interrupts and another, higher priority, for NMIs. Whenever we want to disable interrupts, we mask the standard priority instead so NMIs can still be raised. Some corner cases though still require to actually mask all interrupts effectively disabling the NMI.
Of course, using priority masking instead of PSR.I comes at some cost. On hackbench, I notice a ~6% drop of performance. However, on a kernel build, I do not see significant differences.
Currently, only PPIs and SPIs can be set as NMIs. IPIs being currently hardcoded IRQ numbers, there isn't a generic interface to set SGIs as NMI for now. I don't think there is any reason LPIs should be allowed to be set as NMI as they do not have an active state.
When an NMI is active on a CPU, no other NMI can be triggered on the CPU.
Requirements to use this:
- Have GICv3
- SCR_EL3.FIQ is set to 1 when linux runs
- Select Kernel Feature -> Use ICC system registers for IRQ masking
Quoted 1 line
From my testing with the perf interrupt, there doesn't seem to be any major
issue left (but if you find some, do tell!).
* Patches 1 and 5 are minor utility changes.
* Patches 2 and 3 allows to detect and enable the use of GICv3 system registers during boot time.
* Patch 4 introduces the masking of IRQs using priorities replacing irq disabling.
* Patch 6 add detection of the view linux has on GICv3 priorities, without this we cannot easily mask specific priorities in an accurate manner
* Patch 7 adds the support for NMIs
[1] http://www.spinics.net/lists/arm-kernel/msg525077.html
Thanks,
Julien Thierry
Daniel Thompson (4):
arm64: irqflags: Reorder the fiq & async macros
arm64: cpufeature: Allow early detect of specific features
arm64: alternative: Apply alternatives early in boot process
arm64: irqflags: Use ICC sysregs to implement IRQ masking
Julien Thierry (3):
irqchip/gic: Add functions to access irq priorities
arm64: Detect current view of GIC priorities
arm64: Add support for pseudo-NMIs
Documentation/arm64/booting.txt | 5 + arch/arm64/Kconfig | 15 ++ arch/arm64/include/asm/alternative.h | 1 + arch/arm64/include/asm/arch_gicv3.h | 24 +++ arch/arm64/include/asm/assembler.h | 33 +++- arch/arm64/include/asm/efi.h | 5 + arch/arm64/include/asm/irqflags.h | 143 +++++++++++++++- arch/arm64/include/asm/kvm_host.h | 40 +++++ arch/arm64/include/asm/processor.h | 4 + arch/arm64/include/asm/ptrace.h | 14 +- arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/kernel/alternative.c | 39 ++++- arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/cpufeature.c | 67 +++++--- arch/arm64/kernel/entry.S | 132 ++++++++++++-- arch/arm64/kernel/head.S | 38 +++++ arch/arm64/kernel/process.c | 6 + arch/arm64/kernel/smp.c | 14 ++ arch/arm64/mm/proc.S | 23 +++ drivers/irqchip/irq-gic-common.c | 10 ++ drivers/irqchip/irq-gic-common.h | 2 + drivers/irqchip/irq-gic-v3-its.c | 2 +- drivers/irqchip/irq-gic-v3.c | 304 +++++++++++++++++++++++++++++---- include/linux/interrupt.h | 1 + include/linux/irqchip/arm-gic-common.h | 6 + include/linux/irqchip/arm-gic.h | 5 - 26 files changed, 845 insertions(+), 90 deletions(-)
--
1.9.1
RFC 1/7 arm64: irqflags: Reorder the fiq & async macros
From: Daniel Thompson <[email protected]>
Separate out the local fiq & async macros from the various arch inlines. This makes is easier for us (in a later patch) to provide an alternative implementation of these inlines.
--- arch/arm64/include/asm/irqflags.h | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
arch/arm64/include/asm/irqflags.h +6 -6
@@ -53,12 +53,6 @@ static inline void arch_local_irq_disable(void) : "memory"); } -#define local_fiq_enable () asm("msr daifclr, #1" : : : "memory") -#define local_fiq_disable () asm("msr daifset, #1" : : : "memory") - -#define local_async_enable () asm("msr daifclr, #4" : : : "memory") -#define local_async_disable () asm("msr daifset, #4" : : : "memory") -/* * Save the current interrupt enable state. */
@@ -90,6 +84,12 @@ static inline int arch_irqs_disabled_flags(unsigned long flags)return flags &PSR_I_BIT ; } +#define local_fiq_enable () asm("msr daifclr, #1" : : : "memory") +#define local_fiq_disable () asm("msr daifset, #1" : : : "memory") + +#define local_async_enable () asm("msr daifclr, #4" : : : "memory") +#define local_async_disable () asm("msr daifset, #4" : : : "memory") +/* * save and restore debug state */
--
1.9.1
RFC 2/7 arm64: cpufeature: Allow early detect of specific features
In reply to Julien ThierryFrom: Daniel Thompson <[email protected]>
Currently it is not possible to detect features of the boot CPU until the other CPUs have been brought up.
This prevents us from reacting to features of the boot CPU until fairly late in the boot process. To solve this we allow a subset of features (that are likely to be common to all clusters) to be detected based on the boot CPU alone.
[julien.thierry at arm.com: check non-boot cpu missing early features, avoid
duplicates between early features and normal features, do not move init_cpu_features]
--- arch/arm64/kernel/cpufeature.c | 67 ++++++++++++++++++++++++++++-------------- 1 file changed, 45 insertions(+), 22 deletions(-)
arch/arm64/kernel/cpufeature.c +45 -22
@@ -51,6 +51,8 @@DECLARE_BITMAP (cpu_hwcaps,ARM64_NCAPS );EXPORT_SYMBOL (cpu_hwcaps); +static void __init setup_early_feature_capabilities (void ); +static int dump_cpu_hwcaps (struct notifier_block *self,unsigned long v,void *p) {/* file-wide pr_fmt adds "CPU features: " prefix */
@@ -505,6 +507,7 @@ void __init init_cpu_features(struct cpuinfo_arm64 *info)init_cpu_ftr_reg (SYS_MVFR2_EL1 , info->reg_mvfr2); } +setup_early_feature_capabilities (); }static void update_cpu_ftr_reg (struct arm64_ftr_reg *reg,u64 new)
@@ -796,7 +799,7 @@ static bool has_no_fpsimd(const struct arm64_cpu_capabilities *entry, int __unus ID_AA64PFR0_FP_SHIFT) < 0; } -static const struct arm64_cpu_capabilities arm64_features[] = { +static const struct arm64_cpu_capabilities arm64_early_features[] = { { .desc ="GIC system register CPU interface" , .capability =ARM64_HAS_SYSREG_GIC_CPUIF ,
@@ -807,6 +810,10 @@ static bool has_no_fpsimd(const struct arm64_cpu_capabilities *entry, int __unus .sign = FTR_UNSIGNED, .min_field_value = 1, }, + {} +}; + +static const struct arm64_cpu_capabilities arm64_features[] = { #ifdef CONFIG_ARM64_PAN { .desc ="Privileged Access Never" ,
@@ -1055,6 +1062,29 @@ static inline void set_sys_caps_initialised(void) sys_caps_initialised = true; } +/* Returns false on a capability mismatch */ +static bool +verify_local_cpu_features (const struct arm64_cpu_capabilities *caps) +{ +for (; caps->matches; caps++) { +if (!cpus_have_cap (caps->capability)) +continue ; +/* +* If the new CPU misses an advertised feature, we cannot +* proceed further, park the cpu. +*/ +if (!caps->matches (caps,SCOPE_LOCAL_CPU )) { +pr_crit ("CPU%d: missing feature: %s\n" , +smp_processor_id (), caps->desc); +return false; + } +if (caps->enable) + caps->enable (NULL ); + } + +return true; +} +/* * Check for CPU features that are used in early boot * based on the Boot CPU value.
@@ -1063,6 +1093,9 @@ static void check_early_cpu_features(void) {verify_cpu_run_el ();verify_cpu_asid_bits (); + +if (!verify_local_cpu_features (arm64_early_features)) +cpu_panic_kernel (); } static void
@@ -1077,26 +1110,6 @@ static void check_early_cpu_features(void) } } -static void -verify_local_cpu_features (const struct arm64_cpu_capabilities *caps) -{ -for (; caps->matches; caps++) { -if (!cpus_have_cap (caps->capability)) -continue ; -/* -* If the new CPU misses an advertised feature, we cannot proceed -* further, park the cpu. -*/ -if (!caps->matches (caps,SCOPE_LOCAL_CPU )) { -pr_crit ("CPU%d: missing feature: %s\n" , -smp_processor_id (), caps->desc); -cpu_die_early (); - } -if (caps->enable) - caps->enable (NULL ); - } -} -/* * Run through the enabled system capabilities and enable() it on this CPU. * The capabilities were decided based on the available CPUs at the boot time.
@@ -1108,7 +1121,10 @@ static void check_early_cpu_features(void)static void verify_local_cpu_capabilities (void ) {verify_local_cpu_errata_workarounds (); -verify_local_cpu_features (arm64_features); + +if (!verify_local_cpu_features (arm64_features)) +cpu_die_early (); +verify_local_elf_hwcaps (arm64_elf_hwcaps);if (system_supports_32bit_el0 ())verify_local_elf_hwcaps (compat_elf_hwcaps);
@@ -1134,6 +1150,13 @@ void check_local_cpu_capabilities(void)verify_local_cpu_capabilities (); } +static void __init setup_early_feature_capabilities (void ) +{ +update_cpu_capabilities (arm64_early_features, +"early detected feature:" ); +enable_cpu_capabilities (arm64_early_features); +} +static void __init setup_feature_capabilities (void ) {update_cpu_capabilities (arm64_features,"detected feature:" );
--
1.9.1
RFC 3/7 arm64: alternative: Apply alternatives early in boot process
In reply to Julien ThierryFrom: Daniel Thompson <[email protected]>
Currently alternatives are applied very late in the boot process (and a long time after we enable scheduling). Some alternative sequences, such as those that alter the way CPU context is stored, must be applied much earlier in the boot sequence.
Introduce apply_alternatives_early() to allow some alternatives to be applied immediately after we detect the CPU features of the boot CPU.
--- arch/arm64/include/asm/alternative.h | 1 + arch/arm64/kernel/alternative.c | 39 +++++++++++++++++++++++++++++++++--- arch/arm64/kernel/smp.c | 6 ++++++ 3 files changed, 43 insertions(+), 3 deletions(-)
arch/arm64/include/asm/alternative.h +1 -0
@@ -19,6 +19,7 @@ struct alt_instr {u8 alt_len;/* size of new instruction(s), <= orig_len */ }; +void __init apply_alternatives_early (void );void __init apply_alternatives_all (void );void apply_alternatives (void *start,size_t length);
arch/arm64/kernel/alternative.c +36 -3
@@ -28,6 +28,18 @@#include <asm/sections.h> #include <linux/stop_machine.h> +/* +* early-apply features are detected using only the boot CPU and checked on +* secondary CPUs startup, even then, +* These early-apply features should only include features where we must +* patch the kernel very early in the boot process. +* +* Note that the cpufeature logic *must* be made aware of early-apply +* features to ensure they are reported as enabled without waiting +* for other CPUs to boot. +*/ +#define EARLY_APPLY_FEATURE_MASK BIT(ARM64_HAS_SYSREG_GIC_CPUIF) +#define __ALT_PTR (a,f) ((void *)&(a)->f + (a)->f)#define ALT_ORIG_PTR (a) __ALT_PTR(a, orig_offset)#define ALT_REPL_PTR (a) __ALT_PTR(a, alt_offset)
@@ -105,7 +117,8 @@ static u32 get_alt_insn(struct alt_instr *alt, __le32 *insnptr, __le32 *altinsnpreturn insn; } -static void __apply_alternatives (void *alt_region,bool use_linear_alias) +static void __apply_alternatives (void *alt_region,bool use_linear_alias, +unsigned long feature_mask) {struct alt_instr *alt;struct alt_region *region = alt_region;
@@ -115,6 +128,9 @@ static void __apply_alternatives(void *alt_region, bool use_linear_alias)u32 insn;int i, nr_inst; +if ((BIT (alt->cpufeature) & feature_mask) ==0 ) +continue ; +if (!cpus_have_cap (alt->cpufeature))continue ;
@@ -138,6 +154,21 @@ static void __apply_alternatives(void *alt_region, bool use_linear_alias) }/* +* This is called very early in the boot process (directly after we run +* a feature detect on the boot CPU). No need to worry about other CPUs +* here. +*/ +void apply_alternatives_early (void ) +{ +struct alt_region region = { + .begin = (struct alt_instr *)__alt_instructions, + .end = (struct alt_instr *)__alt_instructions_end, + }; + +__apply_alternatives (®ion, true,EARLY_APPLY_FEATURE_MASK ); +} + +/* * We might be patching the stop_machine state machine, so implement a * really simple polling protocol here. */
@@ -156,7 +187,9 @@ static int __apply_alternatives_multi_stop(void *unused)isb (); }else {BUG_ON (patched); -__apply_alternatives (®ion, true); + +__apply_alternatives (®ion, true, ~EARLY_APPLY_FEATURE_MASK ); +/* Barriers provided by the cache flushing */ WRITE_ONCE (patched,1 ); }
@@ -177,5 +210,5 @@ void apply_alternatives(void *start, size_t length) .end = start + length, }; -__apply_alternatives (®ion, false); +__apply_alternatives (®ion, false,-1 ); }
arch/arm64/kernel/smp.c +6 -0
@@ -455,6 +455,12 @@ void __init smp_prepare_boot_cpu(void)* cpuinfo_store_boot_cpu() above. */ update_cpu_errata_workarounds (); +/* +* We now know enough about the boot CPU to apply the +* alternatives that cannot wait until interrupt handling +* and/or scheduling is enabled. +*/ +apply_alternatives_early (); }static u64 __init of_get_cpu_mpidr (struct device_node *dn)
--
1.9.1
Suzuki K Poulose
reviewerRFC 3/7 arm64: alternative: Apply alternatives early in boot process
Quoted 27 lines · Julien Thierry
From: Daniel Thompson <[email protected]>
Currently alternatives are applied very late in the boot process (and a long time after we enable scheduling). Some alternative sequences, such as those that alter the way CPU context is stored, must be applied much earlier in the boot sequence.
Introduce
apply_alternatives_early()to allow some alternatives to be applied immediately after we detect the CPU features of the boot CPU.Signed-off-by Daniel Thompson <[email protected]>Signed-off-by Julien Thierry <[email protected]>Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>--- arch/arm64/include/asm/alternative.h | 1 + arch/arm64/kernel/alternative.c | 39 +++++++++++++++++++++++++++++++++--- arch/arm64/kernel/smp.c | 6 ++++++ 3 files changed, 43 insertions(+), 3 deletions(-)arch/arm64/kernel/smp.c +0
-0
arch/arm64/kernel/smp.c void __init smp_prepare_boot_cpu(void) · line 464
Quoted 10 lines
arch/arm64/kernel/smp.c +6
-0@@ -455,6 +455,12 @@ void __init smp_prepare_boot_cpu(void)* cpuinfo_store_boot_cpu() above. */ update_cpu_errata_workarounds (); +/* +* We now know enough about the boot CPU to apply the +* alternatives that cannot wait until interrupt handling +* and/or scheduling is enabled. +*/ +apply_alternatives_early (); }
What happens if a secondary CPU, activated at boot doesn't have the "early" capabilities ?
Suzuki
Suzuki K Poulose
reviewerRFC 3/7 arm64: alternative: Apply alternatives early in boot process
Quoted 33 lines · Suzuki K Poulose
Quoted 32 lines · Julien Thierry
From: Daniel Thompson <[email protected]>
Currently alternatives are applied very late in the boot process (and a long time after we enable scheduling). Some alternative sequences, such as those that alter the way CPU context is stored, must be applied much earlier in the boot sequence.
Introduce
apply_alternatives_early()to allow some alternatives to be applied immediately after we detect the CPU features of the boot CPU.Signed-off-by Daniel Thompson <[email protected]>Signed-off-by Julien Thierry <[email protected]>Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>---
? arch/arm64/include/asm/alternative.h |? 1 + ? arch/arm64/kernel/alternative.c????? | 39 +++++++++++++++++++++++++++++++++--- ? arch/arm64/kernel/smp.c????????????? |? 6 ++++++? 3 files changed, 43 insertions(+), 3 deletions(-)
arch/arm64/kernel/smp.c +0
-0@@ -455,6 +455,12 @@ void __init smp_prepare_boot_cpu(void)?????? *
cpuinfo_store_boot_cpu()above.
?????? */
?????update_cpu_errata_workarounds();
+??? /*
+???? * We now know enough about the boot CPU to apply the
Quoted 8 lines
Quoted 5 lines
+???? * alternatives that cannot wait until interrupt handling +???? * and/or scheduling is enabled.
+???? */
+???apply_alternatives_early();
? }What happens if a secondary CPU, activated at boot doesn't have the "early" capabilities ?
Err, ignore this comment. I was somehow missing the patch 2 in the series and after looking that up in the archives, it looks fine.
Cheers
Suzuki
RFC 4/7 arm64: irqflags: Use ICC sysregs to implement IRQ masking
In reply to Julien ThierryFrom: Daniel Thompson <[email protected]>
Currently irqflags is implemented using the PSR's I bit. It is possible to implement irqflags by using the co-processor interface to the GIC. Using the co-processor interface makes it feasible to simulate NMIs using GIC interrupt prioritization.
This patch changes the irqflags macros to modify, save and restore ICC_PMR_EL1. This has a substantial knock on effect for the rest of the kernel. There are four reasons for this:
1. The state of the PMR becomes part of the interrupt context and must be saved and restored during exceptions. It is saved on the stack as part of the saved context when an interrupt/exception is taken.
2. The hardware automatically masks the I bit (at boot, during traps, etc). When the I bit is set by hardware we must add code to switch from I bit masking and PMR masking:
- For IRQs, this is done after the interrupt has been acknowledged avoiding the need to unmask.
- For other exceptions, this is done right after saving the context.
3. Some instructions, such as wfi, require that the PMR not be used for interrupt masking. Before calling these instructions we must switch from PMR masking to I bit masking.
This is also the case when KVM runs a guest, if the CPU receives an interrupt from the host, interrupts must not be masked in PMR otherwise the GIC will not signal it to the CPU.
4. We use the alternatives system to allow a single kernel to boot and be switched to the alternative masking approach at runtime.
[julien.thierry at arm.com: changes reflected in commit,
message, fixes, renaming]
--- arch/arm64/Kconfig | 15 ++++ arch/arm64/include/asm/arch_gicv3.h | 19 +++++ arch/arm64/include/asm/assembler.h | 33 ++++++++- arch/arm64/include/asm/efi.h | 5 ++ arch/arm64/include/asm/irqflags.h | 125 +++++++++++++++++++++++++++++++++ arch/arm64/include/asm/kvm_host.h | 40 +++++++++++ arch/arm64/include/asm/processor.h | 4 ++ arch/arm64/include/asm/ptrace.h | 14 +++- arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/entry.S | 78 ++++++++++++++++---- arch/arm64/kernel/head.S | 38 ++++++++++ arch/arm64/kernel/process.c | 6 ++ arch/arm64/kernel/smp.c | 8 +++ arch/arm64/mm/proc.S | 23 ++++++ drivers/irqchip/irq-gic-v3-its.c | 2 +- drivers/irqchip/irq-gic-v3.c | 82 +++++++++++---------- include/linux/irqchip/arm-gic-common.h | 6 ++ include/linux/irqchip/arm-gic.h | 5 -- 18 files changed, 444 insertions(+), 60 deletions(-)
arch/arm64/Kconfig +15 -0
@@ -803,6 +803,21 @@ config FORCE_MAX_ZONEORDER However for 4K, we choose a higher default value, 11 as opposed to 10, giving us 4M allocations matching the default size used by generic code. +config USE_ICC_SYSREGS_FOR_IRQFLAGS + bool "Use ICC system registers for IRQ masking" + select CONFIG_ARM_GIC_V3 + help + Using the ICC system registers for IRQ masking makes it possible + to simulate NMI on ARM64 systems. This allows several interesting + features (especially debug features) to be used on these systems. + + Say Y here to implement IRQ masking using ICC system + registers when the GIC System Registers are available. The changes + are applied dynamically using the alternatives system so it is safe + to enable this option on systems with older interrupt controllers. + + If unsure, say N + menuconfig ARMV8_DEPRECATED bool "Emulate deprecated/obsolete ARMv8 instructions" depends on COMPAT
arch/arm64/include/asm/arch_gicv3.h +19 -0
@@ -140,5 +140,24 @@ static inline void gic_write_bpr1(u32 val)#define gits_write_vpendbaser (v, c) writeq_relaxed(v, c)#define gits_read_vpendbaser (c) readq_relaxed(c) +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +static inline void gic_start_pmr_masking (void ) +{ +if (static_branch_likely (&cpu_hwcap_keys[ARM64_HAS_SYSREG_GIC_CPUIF ])) { +gic_write_pmr (ICC_PMR_EL1_MASKED ); + asmvolatile ("msr daifclr, #2" : : :"memory" ); + } +} + +static inline void gic_end_pmr_masking (void ) +{ +if (static_branch_likely (&cpu_hwcap_keys[ARM64_HAS_SYSREG_GIC_CPUIF ])) { + asmvolatile ("msr daifset, #2" : : :"memory" ); +gic_write_pmr (ICC_PMR_EL1_UNMASKED ); +dsb (sy); + } +} +#endif +#endif /* __ASSEMBLY__ */ #endif /* __ASM_ARCH_GICV3_H */
arch/arm64/include/asm/assembler.h +30 -3
@@ -23,6 +23,8 @@#ifndef __ASM_ASSEMBLER_H#define __ASM_ASSEMBLER_H +#include <asm/alternative.h> +#include <asm/arch_gicv3.h> #include <asm/asm-offsets.h> #include <asm/cpufeature.h> #include <asm/mmu_context.h>
@@ -34,12 +36,32 @@/* * Enable and disable interrupts. */ - .macro disable_irq + .macro disable_irq, tmp +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + mov \tmp, #ICC_PMR_EL1_MASKED +alternative_if_not ARM64_HAS_SYSREG_GIC_CPUIF msr daifset, #2 +alternative_else + msr_s SYS_ICC_PMR_EL1, \tmp +alternative_endif +#else + msr daifset, #2 +#endif .endm - .macro enable_irq + .macro enable_irq, tmp +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + mov \tmp, #ICC_PMR_EL1_UNMASKED +alternative_if_not ARM64_HAS_SYSREG_GIC_CPUIF msr daifclr, #2 + nop +alternative_else + msr_s SYS_ICC_PMR_EL1, \tmp + dsb sy +alternative_endif +#else + msr daifclr, #2 +#endif .endm .macro save_and_disable_irq, flags
@@ -85,8 +107,13 @@* faster than two daifclr operations, since writes to this register * are self-synchronising. */ - .macro enable_dbg_and_irq + .macro enable_dbg_and_irq, tmp +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + enable_dbg + enable_irq \tmp +#else msr daifclr, #(8 | 2) +#endif .endm/*
arch/arm64/include/asm/efi.h +5 -0
@@ -39,7 +39,12 @@efi_virtmap_unload (); \ }) +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +#define ARCH_EFI_IRQ_FLAGS_MASK \ + (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT | ARCH_FLAG_PMR_EN) +#else #define ARCH_EFI_IRQ_FLAGS_MASK (PSR_D_BIT | PSR_A_BIT | PSR_I_BIT | PSR_F_BIT) +#endif /* arch specific definitions used by the stub code */
arch/arm64/include/asm/irqflags.h +125 -0
138 lines not on this page · raw message
arch/arm64/include/asm/kvm_host.h +40 -0
@@ -38,6 +38,10 @@#include <kvm/arm_arch_timer.h> #include <kvm/arm_pmu.h> +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +#include <asm/arch_gicv3.h> +#endif +#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS#define KVM_VCPU_MAX_FEATURES 4
@@ -332,7 +336,30 @@ int kvm_unmap_hva_range(struct kvm *kvm,void kvm_arm_resume_guest (struct kvm *kvm);u64 __kvm_call_hyp (void *hypfn, ...); + +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +/* +* If we use PMR to disable interrupts, interrupts happening while the +* guest is executing will not be signaled to the host by the GIC (and +* any call to a waiting kvm_kick_many_cpus will likely cause a +* deadlock). +* We need to switch back to using PSR.I. +*/ +#define kvm_call_hyp (f, ...) \ + ({ \ + u64 res; \ + unsigned long flags; \ + \ + flags = arch_local_irq_save(); \ + gic_end_pmr_masking(); \ + res = __kvm_call_hyp(kvm_ksym_ref(f), ##__VA_ARGS__); \ + gic_start_pmr_masking(); \ + arch_local_irq_restore(flags); \ + res; \ + }) +#else #define kvm_call_hyp (f, ...) __kvm_call_hyp(kvm_ksym_ref(f), ##__VA_ARGS__) +#endif void force_vm_exit (const cpumask_t *mask);void kvm_mmu_wp_memory_region (struct kvm *kvm,int slot);
@@ -349,6 +376,9 @@ static inline void __cpu_init_hyp_mode(phys_addr_t pgd_ptr,unsigned long hyp_stack_ptr, unsigned long vector_ptr) { +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + unsigned long flags; +#endif /* * Call initialization code, and switch to the full blown HYP code. * If the cpucaps haven't been finalized yet, something has gone very
@@ -356,7 +386,17 @@ static inline void __cpu_init_hyp_mode(phys_addr_t pgd_ptr,* cpus_have_const_cap() wrapper. */ BUG_ON (!static_branch_likely (&arm64_const_caps_ready)); +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + flags =arch_local_irq_save (); +gic_end_pmr_masking (); +#endif +__kvm_call_hyp ((void *)pgd_ptr, hyp_stack_ptr, vector_ptr); + +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +gic_start_pmr_masking (); +arch_local_irq_restore (flags); +#endif }static inline void kvm_arch_hardware_unsetup (void ) {}
arch/arm64/include/asm/processor.h +4 -0
@@ -114,6 +114,10 @@ static inline void start_thread_common(struct pt_regs *regs, unsigned long pc)memset (regs,0 ,sizeof (*regs));forget_syscall (regs); regs->pc = pc; +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +/* Have IRQs enabled by default */ + regs->pmr_save =ICC_PMR_EL1_UNMASKED ; +#endif } static inline void start_thread(struct pt_regs *regs, unsigned long pc,
arch/arm64/include/asm/ptrace.h +13 -1
@@ -25,6 +25,12 @@#define CurrentEL_EL1 (1 << 2)#define CurrentEL_EL2 (2 << 2) +/* PMR values used to mask/unmask interrupts */ +#define ICC_PMR_EL1_EN_SHIFT 6 +#define ICC_PMR_EL1_EN_BIT (1 << ICC_PMR_EL1_EN_SHIFT) // PMR IRQ enable +#define ICC_PMR_EL1_UNMASKED 0xf0 +#define ICC_PMR_EL1_MASKED (ICC_PMR_EL1_UNMASKED ^ ICC_PMR_EL1_EN_BIT) +/* AArch32-specific ptrace requests */ #define COMPAT_PTRACE_GETREGS 12#define COMPAT_PTRACE_SETREGS 13
@@ -136,7 +142,7 @@ struct pt_regs {#endif u64 orig_addr_limit; -u64 unused;// maintain 16 byte alignment +u64 pmr_save;u64 stackframe[2 ]; };
@@ -171,8 +177,14 @@ static inline void forget_syscall(struct pt_regs *regs)#define processor_mode (regs) \ ((regs)->pstate & PSR_MODE_MASK) +#ifndef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS #define interrupts_enabled (regs) \ (!((regs)->pstate & PSR_I_BIT)) +#else +#define interrupts_enabled (regs) \ + ((!((regs)->pstate & PSR_I_BIT)) && \ + ((regs)->pmr_save & ICC_PMR_EL1_EN_BIT)) +#endif #define fast_interrupts_enabled (regs) \ (!((regs)->pstate & PSR_F_BIT))
arch/arm64/kernel/asm-offsets.c +1 -0
@@ -75,6 +75,7 @@ int main(void)DEFINE (S_ORIG_X0 , offsetof(struct pt_regs , orig_x0));DEFINE (S_SYSCALLNO , offsetof(struct pt_regs , syscallno));DEFINE (S_ORIG_ADDR_LIMIT , offsetof(struct pt_regs , orig_addr_limit)); +DEFINE (S_PMR_SAVE , offsetof(struct pt_regs , pmr_save));DEFINE (S_STACKFRAME , offsetof(struct pt_regs , stackframe));DEFINE (S_FRAME_SIZE ,sizeof (struct pt_regs ));BLANK ();
arch/arm64/kernel/entry.S +66 -12
192 lines not on this page · raw message
arch/arm64/kernel/head.S +38 -0
@@ -545,6 +545,44 @@ set_cpu_boot_mode_flag: ret ENDPROC(set_cpu_boot_mode_flag) +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +/* + * void maybe_switch_to_sysreg_gic_cpuif(void) + * + * Enable interrupt controller system register access if this feature + * has been detected by the alternatives system. + * + * Before we jump into generic code we must enable interrupt controller system + * register access because this is required by the irqflags macros. We must + * also mask interrupts at the PMR and unmask them within the PSR. That leaves + * us set up and ready for the kernel to make its first call to + * arch_local_irq_enable(). + + * + */ +ENTRY(maybe_switch_to_sysreg_gic_cpuif) +alternative_if_not ARM64_HAS_SYSREG_GIC_CPUIF + b 1f +alternative_else + mrs_s x0, SYS_ICC_SRE_EL1 +alternative_endif + orr x0, x0, #1 + msr_s SYS_ICC_SRE_EL1, x0 // Set ICC_SRE_EL1.SRE==1 + isb // Make sure SRE is now set + mrs x0, daif + tbz x0, #7, no_mask_pmr // Are interrupts on? + mov x0, ICC_PMR_EL1_MASKED + msr_s SYS_ICC_PMR_EL1, x0 // Prepare for unmask of I bit + msr daifclr, #2 // Clear the I bit + b 1f +no_mask_pmr: + mov x0, ICC_PMR_EL1_UNMASKED + msr_s SYS_ICC_PMR_EL1, x0 +1: + ret +ENDPROC(maybe_switch_to_sysreg_gic_cpuif) +#endif + /* * These values are written with the MMU off, but read with the MMU on. * Writers will invalidate the corresponding address, discarding up to a
arch/arm64/kernel/process.c +6 -0
@@ -65,6 +65,8 @@EXPORT_SYMBOL (__stack_chk_guard);#endif +#include <asm/arch_gicv3.h> +/* * Function pointers to optional machine specific functions */
@@ -191,6 +193,7 @@ void __show_regs(struct pt_regs *regs)printk ("pc : [<%016llx>] lr : [<%016llx>] pstate: %08llx\n" , regs->pc, lr, regs->pstate);printk ("sp : %016llx\n" , sp); +printk ("pmr_save: %08llx\n" , regs->pmr_save); i = top_reg;
@@ -284,6 +287,9 @@ int copy_thread(unsigned long clone_flags, unsigned long stack_start, }else {memset (childregs,0 ,sizeof (struct pt_regs )); childregs->pstate = PSR_MODE_EL1h; +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + childregs->pmr_save =ICC_PMR_EL1_UNMASKED ; +#endif if (IS_ENABLED (CONFIG_ARM64_UAO ) &&cpus_have_const_cap (ARM64_HAS_UAO )) childregs->pstate |=PSR_UAO_BIT ;
arch/arm64/kernel/smp.c +8 -0
@@ -219,6 +219,8 @@ asmlinkage void secondary_start_kernel(void)struct mm_struct *mm = &init_mm;unsigned int cpu; +maybe_switch_to_sysreg_gic_cpuif (); + cpu =task_cpu (current);set_my_cpu_offset (per_cpu_offset (cpu));
@@ -461,6 +463,12 @@ void __init smp_prepare_boot_cpu(void)* and/or scheduling is enabled. */ apply_alternatives_early (); + +/* +* Conditionally switch to GIC PMR for interrupt masking (this +* will be a nop if we are using normal interrupt masking) +*/ +maybe_switch_to_sysreg_gic_cpuif (); }static u64 __init of_get_cpu_mpidr (struct device_node *dn)
arch/arm64/mm/proc.S +23 -0
@@ -20,6 +20,7 @@ #include <linux/init.h> #include <linux/linkage.h> +#include <linux/irqchip/arm-gic-v3.h> #include <asm/assembler.h> #include <asm/asm-offsets.h> #include <asm/hwcap.h>
@@ -47,11 +48,33 @@ * cpu_do_idle() * * Idle the processor (wait for interrupt). + * + * If CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS is set we must do additional + * work to ensure that interrupts are not masked at the PMR (because the + * core will not wake up if we block the wake up signal in the interrupt + * controller). */ ENTRY(cpu_do_idle) +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +alternative_if_not ARM64_HAS_SYSREG_GIC_CPUIF +#endif + dsb sy // WFI may enter a low-power mode + wfi + ret +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +alternative_else + mrs x0, daif // save I bit + msr daifset, #2 // set I bit + mrs_s x1, SYS_ICC_PMR_EL1 // save PMR +alternative_endif + mov x2, #ICC_PMR_EL1_UNMASKED + msr_s SYS_ICC_PMR_EL1, x2 // unmask at PMR dsb sy // WFI may enter a low-power mode wfi + msr_s SYS_ICC_PMR_EL1, x1 // restore PMR + msr daif, x0 // restore I bit ret +#endif ENDPROC(cpu_do_idle) #ifdef CONFIG_CPU_PM
drivers/irqchip/irq-gic-v3-its.c +1 -1
@@ -60,7 +60,7 @@#define LPI_PROPBASE_SZ ALIGN(BIT(LPI_NRBITS), SZ_64K)#define LPI_PENDBASE_SZ ALIGN(BIT(LPI_NRBITS) / 8, SZ_64K) -#define LPI_PROP_DEFAULT_PRIO 0xa0 +#define LPI_PROP_DEFAULT_PRIO GICD_INT_DEF_PRI/* * Collection structure - just an ID, and a redistributor address to
drivers/irqchip/irq-gic-v3.c +44 -38
109 lines not on this page · raw message
include/linux/irqchip/arm-gic-common.h +6 -0
@@ -13,6 +13,12 @@#include <linux/types.h> #include <linux/ioport.h> +#define GICD_INT_DEF_PRI 0xc0 +#define GICD_INT_DEF_PRI_X4 ((GICD_INT_DEF_PRI << 24) |\ + (GICD_INT_DEF_PRI << 16) |\ + (GICD_INT_DEF_PRI << 8) |\ + GICD_INT_DEF_PRI) +enum gic_type {GIC_V2 ,GIC_V3 ,
include/linux/irqchip/arm-gic.h +0 -5
@@ -65,11 +65,6 @@#define GICD_INT_EN_CLR_X32 0xffffffff#define GICD_INT_EN_SET_SGI 0x0000ffff#define GICD_INT_EN_CLR_PPI 0xffff0000 -#define GICD_INT_DEF_PRI 0xa0 -#define GICD_INT_DEF_PRI_X4 ((GICD_INT_DEF_PRI << 24) |\ - (GICD_INT_DEF_PRI << 16) |\ - (GICD_INT_DEF_PRI << 8) |\ - GICD_INT_DEF_PRI)#define GICH_HCR 0x0#define GICH_VTR 0x4
--
1.9.1
RFC 4/7 arm64: irqflags: Use ICC sysregs to implement IRQ masking
Quoted 35 lines · Julien Thierry
From: Daniel Thompson <[email protected]>
Currently irqflags is implemented using the PSR's I bit. It is possible to implement irqflags by using the co-processor interface to the GIC. Using the co-processor interface makes it feasible to simulate NMIs using GIC interrupt prioritization.
This patch changes the irqflags macros to modify, save and restore
ICC_PMR_EL1. This has a substantial knock on effect for the rest of the kernel. There are four reasons for this:1. The state of the PMR becomes part of the interrupt context and must be saved and restored during exceptions. It is saved on the stack as part of the saved context when an interrupt/exception is taken.
2. The hardware automatically masks the I bit (at boot, during traps, etc). When the I bit is set by hardware we must add code to switch from I bit masking and PMR masking:
- For IRQs, this is done after the interrupt has been acknowledged avoiding the need to unmask.
- For other exceptions, this is done right after saving the context.3. Some instructions, such as wfi, require that the PMR not be used for interrupt masking. Before calling these instructions we must switch from PMR masking to I bit masking.
This is also the case when KVM runs a guest, if the CPU receives an interrupt from the host, interrupts must not be masked in PMR otherwise the GIC will not signal it to the CPU.4. We use the alternatives system to allow a single kernel to boot and be switched to the alternative masking approach at runtime.
Signed-off-by Daniel Thompson <[email protected]>[julien.thierry at arm.com: changes reflected in commit,
message, fixes, renaming]
Quoted 7 lines
Signed-off-by Julien Thierry <[email protected]>Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Christoffer Dall <[email protected]>Cc Marc Zyngier <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Jason Cooper <[email protected]>
---
I just gave a quick look to the KVM part here but didn't get into whether this rather invasive change is warranted or not (it's definitely entertaining though).
[...]
Quoted 16 lines
arch/arm64/include/asm/kvm_host.h +4
-0@@ -38,6 +38,10 @@#include <kvm/arm_arch_timer.h> #include <kvm/arm_pmu.h> +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +#include <asm/arch_gicv3.h> +#endif +#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS#define KVM_VCPU_MAX_FEATURES 4
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 342
Quoted 7 lines
arch/arm64/include/asm/kvm_host.h +4
-0@@ -332,7 +336,30 @@ int kvm_unmap_hva_range(struct kvm *kvm,void kvm_arm_resume_guest (struct kvm *kvm);u64 __kvm_call_hyp (void *hypfn, ...); + +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +/* +* If we use PMR to disable interrupts, interrupts happening while the
I would describe this as 'masking' interrupts, as opposed to 'disabling' interrupts, in general, and probably we can be more precise by categorizing interrupts as 'normal' vs. 'priority (NMIs)' or something like that.
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 343
Quoted 1 line
arch/arm64/include/asm/kvm_host.h +1
-0+* guest is executing will not be signaled to the host by the GIC (and
signaled to the CPU. (Whether this then causes an exception to EL2 depends on the HCR_EL2.IMO flag and what KVM does with that exception).
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 345
Quoted 2 lines
arch/arm64/include/asm/kvm_host.h +2
-0+* any call to a waiting kvm_kick_many_cpus will likely cause a +* deadlock).
This kick thing is an unccessary specific example to bring out here, I don't think it adds to the general understanding.
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 358
Quoted 13 lines
arch/arm64/include/asm/kvm_host.h +13
-0+* We need to switch back to using PSR.I. +*/ +#define kvm_call_hyp (f, ...) \ + ({ \ + u64 res; \ + unsigned long flags; \ + \ + flags = arch_local_irq_save(); \ + gic_end_pmr_masking(); \ + res = __kvm_call_hyp(kvm_ksym_ref(f), ##__VA_ARGS__); \ + gic_start_pmr_masking(); \ + arch_local_irq_restore(flags); \ + res; \
This is in the critical path, so ideally this could instead be done inside the function, because the trap will set PSTATE.I already, so it should be possible to reduce this to a set of save/clear/restore operations after __kvm_call_hyp.
Quoted 22 lines
arch/arm64/include/asm/kvm_host.h +7
-0+ }) +#else #define kvm_call_hyp (f, ...) __kvm_call_hyp(kvm_ksym_ref(f), ##__VA_ARGS__) +#endif void force_vm_exit (const cpumask_t *mask);void kvm_mmu_wp_memory_region (struct kvm *kvm,int slot);@@ -349,6 +376,9 @@ static inline void __cpu_init_hyp_mode(phys_addr_t pgd_ptr,unsigned long hyp_stack_ptr, unsigned long vector_ptr) { +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + unsigned long flags; +#endif /* * Call initialization code, and switch to the full blown HYP code. * If the cpucaps haven't been finalized yet, something has gone very @@ -356,7 +386,17 @@ static inline void __cpu_init_hyp_mode(phys_addr_t pgd_ptr,* cpus_have_const_cap() wrapper. */ BUG_ON (!static_branch_likely (&arm64_const_caps_ready)); +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS
arch/arm64/include/asm/kvm_host.h static inline void __cpu_init_hyp_mode(phys_addr_t pgd_ptr, · line 399
Quoted 10 lines
arch/arm64/include/asm/kvm_host.h +9
-0+ flags =arch_local_irq_save (); +gic_end_pmr_masking (); +#endif +__kvm_call_hyp ((void *)pgd_ptr, hyp_stack_ptr, vector_ptr); + +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +gic_start_pmr_masking (); +arch_local_irq_restore (flags); +#endif
I don't think you need any of this, because I don't think you'd ever need to handle interrupts while initializing hyp mode.
arch/arm64/include/asm/kvm_host.h static inline void __cpu_init_hyp_mode(phys_addr_t pgd_ptr, · line 401
Quoted 2 lines
arch/arm64/include/asm/kvm_host.h +0
-0}
Thanks,
-Christoffer
RFC 4/7 arm64: irqflags: Use ICC sysregs to implement IRQ masking
Quoted 43 lines · Christoffer Dall
Quoted 35 lines · Julien Thierry
From: Daniel Thompson <[email protected]>
Currently irqflags is implemented using the PSR's I bit. It is possible to implement irqflags by using the co-processor interface to the GIC. Using the co-processor interface makes it feasible to simulate NMIs using GIC interrupt prioritization.
This patch changes the irqflags macros to modify, save and restore
ICC_PMR_EL1. This has a substantial knock on effect for the rest of the kernel. There are four reasons for this:1. The state of the PMR becomes part of the interrupt context and must be saved and restored during exceptions. It is saved on the stack as part of the saved context when an interrupt/exception is taken.
2. The hardware automatically masks the I bit (at boot, during traps, etc). When the I bit is set by hardware we must add code to switch from I bit masking and PMR masking:
- For IRQs, this is done after the interrupt has been acknowledged avoiding the need to unmask.
- For other exceptions, this is done right after saving the context.3. Some instructions, such as wfi, require that the PMR not be used for interrupt masking. Before calling these instructions we must switch from PMR masking to I bit masking.
This is also the case when KVM runs a guest, if the CPU receives an interrupt from the host, interrupts must not be masked in PMR otherwise the GIC will not signal it to the CPU.4. We use the alternatives system to allow a single kernel to boot and be switched to the alternative masking approach at runtime.
Signed-off-by Daniel Thompson <[email protected]>[julien.thierry at arm.com: changes reflected in commit,
message, fixes, renaming]Quoted 7 lines
Signed-off-by Julien Thierry <[email protected]>Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Christoffer Dall <[email protected]>Cc Marc Zyngier <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Jason Cooper <[email protected]>
Quoted 6 lines
---
I just gave a quick look to the KVM part here but didn't get into whether this rather invasive change is warranted or not (it's definitely entertaining though).
I appreciate you looking into this, it wasn't very clear to me how to deal with that for KVM.
Quoted 21 lines
[...]
Quoted 18 lines
arch/arm64/include/asm/kvm_host.h +4
-0@@ -38,6 +38,10 @@#include <kvm/arm_arch_timer.h> #include <kvm/arm_pmu.h> +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +#include <asm/arch_gicv3.h> +#endif +#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS#define KVM_VCPU_MAX_FEATURES 4@@ -332,7 +336,30 @@ int kvm_unmap_hva_range(struct kvm *kvm,void kvm_arm_resume_guest (struct kvm *kvm);
Quoted 11 lines
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 342
Quoted 5 lines
arch/arm64/include/asm/kvm_host.h +4
-0u64 __kvm_call_hyp (void *hypfn, ...); + +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +/* +* If we use PMR to disable interrupts, interrupts happening while the I would describe this as 'masking' interrupts, as opposed to 'disabling' interrupts, in general, and probably we can be more precise by categorizing interrupts as 'normal' vs. 'priority (NMIs)' or something like that.
Right, I'll do that.
Quoted 5 lines
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 343
Quoted 1 line
arch/arm64/include/asm/kvm_host.h +1
-0+* guest is executing will not be signaled to the host by the GIC (and signaled to the CPU. (Whether this then causes an exception to EL2 depends on the HCR_EL2.IMO flag and what KVM does with that exception).
True, I'll fix that.
Quoted 6 lines
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 345
Quoted 2 lines
arch/arm64/include/asm/kvm_host.h +2
-0+* any call to a waiting kvm_kick_many_cpus will likely cause a +* deadlock). This kick thing is an unccessary specific example to bring out here, I don't think it adds to the general understanding.
Yes, makes sense.
Quoted 10 lines
Quoted 10 lines
arch/arm64/include/asm/kvm_host.h +10
-0+* We need to switch back to using PSR.I. +*/ +#define kvm_call_hyp (f, ...) \ + ({ \ + u64 res; \ + unsigned long flags; \ + \ + flags = arch_local_irq_save(); \ + gic_end_pmr_masking(); \ + res = __kvm_call_hyp(kvm_ksym_ref(f), ##__VA_ARGS__); \
Quoted 8 lines
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 358
Quoted 3 lines
arch/arm64/include/asm/kvm_host.h +3
-0+gic_start_pmr_masking (); \ +arch_local_irq_restore (flags); \ + res; \This is in the critical path, so ideally this could instead be done inside the function, because the trap will set PSTATE.I already, so it should be possible to reduce this to a set of save/clear/restore operations after
__kvm_call_hyp.
Hmmm, you are talking about the function called by __kvm_call_hyp, right?
But this means we'll need to add the save/clear/restore to each function that can be called by __kvm_call_hyp, no?
Also in the VHE case we don't have the trap setting the PSTATE.I (the code calling kvm_call_hyp disables interrupts before entering the guest, but now the function disabling interrupts is just masking them with PMR).
Quoted 28 lines
Quoted 28 lines
arch/arm64/include/asm/kvm_host.h +11
-0+ }) +#else #define kvm_call_hyp (f, ...) __kvm_call_hyp(kvm_ksym_ref(f), ##__VA_ARGS__) +#endif void force_vm_exit (const cpumask_t *mask);void kvm_mmu_wp_memory_region (struct kvm *kvm,int slot);@@ -349,6 +376,9 @@ static inline void __cpu_init_hyp_mode(phys_addr_t pgd_ptr,unsigned long hyp_stack_ptr, unsigned long vector_ptr) { +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + unsigned long flags; +#endif /* * Call initialization code, and switch to the full blown HYP code. * If the cpucaps haven't been finalized yet, something has gone very @@ -356,7 +386,17 @@ static inline void __cpu_init_hyp_mode(phys_addr_t pgd_ptr,* cpus_have_const_cap() wrapper. */ BUG_ON (!static_branch_likely (&arm64_const_caps_ready)); +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + flags =arch_local_irq_save (); +gic_end_pmr_masking (); +#endif +__kvm_call_hyp ((void *)pgd_ptr, hyp_stack_ptr, vector_ptr);
Quoted 9 lines
arch/arm64/include/asm/kvm_host.h static inline void __cpu_init_hyp_mode(phys_addr_t pgd_ptr, · line 399
Quoted 5 lines
arch/arm64/include/asm/kvm_host.h +5
-0+ +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +gic_start_pmr_masking (); +arch_local_irq_restore (flags); +#endif I don't think you need any of this, because I don't think you'd ever need to handle interrupts while initializing hyp mode.
I was more worried about getting an "NMI" during the EL2 initialisation rather than missing the masked interrupts. Do you know whether this might be an issue here or not?
Thanks,
Signature · 1 line
Julien Thierry
RFC 4/7 arm64: irqflags: Use ICC sysregs to implement IRQ masking
Quoted 127 lines · Julien Thierry
Quoted 43 lines · Christoffer Dall
Quoted 35 lines · Julien Thierry
From: Daniel Thompson <[email protected]>
Currently irqflags is implemented using the PSR's I bit. It is possible to implement irqflags by using the co-processor interface to the GIC. Using the co-processor interface makes it feasible to simulate NMIs using GIC interrupt prioritization.
This patch changes the irqflags macros to modify, save and restore
ICC_PMR_EL1. This has a substantial knock on effect for the rest of the kernel. There are four reasons for this:1. The state of the PMR becomes part of the interrupt context and must be saved and restored during exceptions. It is saved on the stack as part of the saved context when an interrupt/exception is taken.
2. The hardware automatically masks the I bit (at boot, during traps, etc). When the I bit is set by hardware we must add code to switch from I bit masking and PMR masking:
- For IRQs, this is done after the interrupt has been acknowledged avoiding the need to unmask.
- For other exceptions, this is done right after saving the context.3. Some instructions, such as wfi, require that the PMR not be used for interrupt masking. Before calling these instructions we must switch from PMR masking to I bit masking.
This is also the case when KVM runs a guest, if the CPU receives an interrupt from the host, interrupts must not be masked in PMR otherwise the GIC will not signal it to the CPU.4. We use the alternatives system to allow a single kernel to boot and be switched to the alternative masking approach at runtime.
Signed-off-by Daniel Thompson <[email protected]>[julien.thierry at arm.com: changes reflected in commit,
message, fixes, renaming]Quoted 7 lines
Signed-off-by Julien Thierry <[email protected]>Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Christoffer Dall <[email protected]>Cc Marc Zyngier <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Jason Cooper <[email protected]>Quoted 6 lines
---
I just gave a quick look to the KVM part here but didn't get into whether this rather invasive change is warranted or not (it's definitely entertaining though).
I appreciate you looking into this, it wasn't very clear to me how to deal with that for KVM.
Quoted 21 lines
[...]
Quoted 18 lines
arch/arm64/include/asm/kvm_host.h +4
-0@@ -38,6 +38,10 @@#include <kvm/arm_arch_timer.h> #include <kvm/arm_pmu.h> +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +#include <asm/arch_gicv3.h> +#endif +#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS#define KVM_VCPU_MAX_FEATURES 4@@ -332,7 +336,30 @@ int kvm_unmap_hva_range(struct kvm *kvm,void kvm_arm_resume_guest (struct kvm *kvm);Quoted 11 lines
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 342
Quoted 5 lines
arch/arm64/include/asm/kvm_host.h +4
-0u64 __kvm_call_hyp (void *hypfn, ...); + +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +/* +* If we use PMR to disable interrupts, interrupts happening while the I would describe this as 'masking' interrupts, as opposed to 'disabling' interrupts, in general, and probably we can be more precise by categorizing interrupts as 'normal' vs. 'priority (NMIs)' or something like that.
Right, I'll do that.
Quoted 5 lines
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 343
Quoted 1 line
arch/arm64/include/asm/kvm_host.h +1
-0+* guest is executing will not be signaled to the host by the GIC (and signaled to the CPU. (Whether this then causes an exception to EL2 depends on the HCR_EL2.IMO flag and what KVM does with that exception).
True, I'll fix that.
Quoted 6 lines
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 345
Quoted 2 lines
arch/arm64/include/asm/kvm_host.h +2
-0+* any call to a waiting kvm_kick_many_cpus will likely cause a +* deadlock). This kick thing is an unccessary specific example to bring out here, I don't think it adds to the general understanding.
Yes, makes sense.
Quoted 10 lines
Quoted 10 lines
arch/arm64/include/asm/kvm_host.h +10
-0+* We need to switch back to using PSR.I. +*/ +#define kvm_call_hyp (f, ...) \ + ({ \ + u64 res; \ + unsigned long flags; \ + \ + flags = arch_local_irq_save(); \ + gic_end_pmr_masking(); \ + res = __kvm_call_hyp(kvm_ksym_ref(f), ##__VA_ARGS__); \Quoted 8 lines
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 358
Quoted 3 lines
arch/arm64/include/asm/kvm_host.h +3
-0+gic_start_pmr_masking (); \ +arch_local_irq_restore (flags); \ + res; \This is in the critical path, so ideally this could instead be done inside the function, because the trap will set PSTATE.I already, so it should be possible to reduce this to a set of save/clear/restore operations after
__kvm_call_hyp.
Quoted 4 lines
Hmmm, you are talking about the function called by
__kvm_call_hyp, right?But this means we'll need to add the save/clear/restore to each function that can be called by
__kvm_call_hyp, no?
No, you only need to do this in the case where you run the guedt, __kvm_vcpu_run, because all the other callers don't need to be able to take interrupts as they are under control of the host.
Quoted 5 lines
Also in the VHE case we don't have the trap setting the PSTATE.I (the code calling
kvm_call_hypdisables interrupts before entering the guest, but now the function disabling interrupts is just masking them with PMR).
Yes, for VHE you'd have to do something more. One option would be to disable interrupts using PSTATE.I in kvm_arch_vcpu_ioctl_run() for VHE, another option is to simply mask interrupts in the PSTATE only for VHE in __kvm_vcpu_run.
Quoted 38 lines
Quoted 28 lines
Quoted 28 lines
arch/arm64/include/asm/kvm_host.h +11
-0+ }) +#else #define kvm_call_hyp (f, ...) __kvm_call_hyp(kvm_ksym_ref(f), ##__VA_ARGS__) +#endif void force_vm_exit (const cpumask_t *mask);void kvm_mmu_wp_memory_region (struct kvm *kvm,int slot);@@ -349,6 +376,9 @@ static inline void __cpu_init_hyp_mode(phys_addr_t pgd_ptr,unsigned long hyp_stack_ptr, unsigned long vector_ptr) { +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + unsigned long flags; +#endif /* * Call initialization code, and switch to the full blown HYP code. * If the cpucaps haven't been finalized yet, something has gone very @@ -356,7 +386,17 @@ static inline void __cpu_init_hyp_mode(phys_addr_t pgd_ptr,* cpus_have_const_cap() wrapper. */ BUG_ON (!static_branch_likely (&arm64_const_caps_ready)); +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + flags =arch_local_irq_save (); +gic_end_pmr_masking (); +#endif +__kvm_call_hyp ((void *)pgd_ptr, hyp_stack_ptr, vector_ptr);Quoted 9 lines
arch/arm64/include/asm/kvm_host.h static inline void __cpu_init_hyp_mode(phys_addr_t pgd_ptr, · line 399
Quoted 5 lines
arch/arm64/include/asm/kvm_host.h +5
-0+ +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +gic_start_pmr_masking (); +arch_local_irq_restore (flags); +#endif I don't think you need any of this, because I don't think you'd ever need to handle interrupts while initializing hyp mode.
Quoted 4 lines
I was more worried about getting an "NMI" during the EL2 initialisation rather than missing the masked interrupts. Do you know whether this might be an issue here or not?
For non-VHE it won't be an issue, because making a hyp call will mask interrupts using PSTATE.I, and for VHE we don't do any work here, so it's not an issue there either.
Thanks,
-Christoffer
RFC 4/7 arm64: irqflags: Use ICC sysregs to implement IRQ masking
Quoted 143 lines · Christoffer Dall
Quoted 127 lines · Julien Thierry
Quoted 43 lines · Christoffer Dall
Quoted 35 lines · Julien Thierry
From: Daniel Thompson <[email protected]>
Currently irqflags is implemented using the PSR's I bit. It is possible to implement irqflags by using the co-processor interface to the GIC. Using the co-processor interface makes it feasible to simulate NMIs using GIC interrupt prioritization.
This patch changes the irqflags macros to modify, save and restore
ICC_PMR_EL1. This has a substantial knock on effect for the rest of the kernel. There are four reasons for this:1. The state of the PMR becomes part of the interrupt context and must be saved and restored during exceptions. It is saved on the stack as part of the saved context when an interrupt/exception is taken.
2. The hardware automatically masks the I bit (at boot, during traps, etc). When the I bit is set by hardware we must add code to switch from I bit masking and PMR masking:
- For IRQs, this is done after the interrupt has been acknowledged avoiding the need to unmask.
- For other exceptions, this is done right after saving the context.3. Some instructions, such as wfi, require that the PMR not be used for interrupt masking. Before calling these instructions we must switch from PMR masking to I bit masking.
This is also the case when KVM runs a guest, if the CPU receives an interrupt from the host, interrupts must not be masked in PMR otherwise the GIC will not signal it to the CPU.4. We use the alternatives system to allow a single kernel to boot and be switched to the alternative masking approach at runtime.
Signed-off-by Daniel Thompson <[email protected]>[julien.thierry at arm.com: changes reflected in commit,
message, fixes, renaming]Quoted 7 lines
Signed-off-by Julien Thierry <[email protected]>Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Christoffer Dall <[email protected]>Cc Marc Zyngier <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Jason Cooper <[email protected]>Quoted 6 lines
---
I just gave a quick look to the KVM part here but didn't get into whether this rather invasive change is warranted or not (it's definitely entertaining though).
I appreciate you looking into this, it wasn't very clear to me how to deal with that for KVM.
Quoted 21 lines
[...]
Quoted 18 lines
arch/arm64/include/asm/kvm_host.h +4
-0@@ -38,6 +38,10 @@#include <kvm/arm_arch_timer.h> #include <kvm/arm_pmu.h> +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +#include <asm/arch_gicv3.h> +#endif +#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS#define KVM_VCPU_MAX_FEATURES 4@@ -332,7 +336,30 @@ int kvm_unmap_hva_range(struct kvm *kvm,void kvm_arm_resume_guest (struct kvm *kvm);Quoted 11 lines
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 342
Quoted 5 lines
arch/arm64/include/asm/kvm_host.h +4
-0u64 __kvm_call_hyp (void *hypfn, ...); + +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +/* +* If we use PMR to disable interrupts, interrupts happening while the I would describe this as 'masking' interrupts, as opposed to 'disabling' interrupts, in general, and probably we can be more precise by categorizing interrupts as 'normal' vs. 'priority (NMIs)' or something like that.
Right, I'll do that.
Quoted 5 lines
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 343
Quoted 1 line
arch/arm64/include/asm/kvm_host.h +1
-0+* guest is executing will not be signaled to the host by the GIC (and signaled to the CPU. (Whether this then causes an exception to EL2 depends on the HCR_EL2.IMO flag and what KVM does with that exception).
True, I'll fix that.
Quoted 6 lines
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 345
Quoted 2 lines
arch/arm64/include/asm/kvm_host.h +2
-0+* any call to a waiting kvm_kick_many_cpus will likely cause a +* deadlock). This kick thing is an unccessary specific example to bring out here, I don't think it adds to the general understanding.
Yes, makes sense.
Quoted 10 lines
Quoted 10 lines
arch/arm64/include/asm/kvm_host.h +10
-0+* We need to switch back to using PSR.I. +*/ +#define kvm_call_hyp (f, ...) \ + ({ \ + u64 res; \ + unsigned long flags; \ + \ + flags = arch_local_irq_save(); \ + gic_end_pmr_masking(); \ + res = __kvm_call_hyp(kvm_ksym_ref(f), ##__VA_ARGS__); \Quoted 8 lines
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 358
Quoted 3 lines
arch/arm64/include/asm/kvm_host.h +3
-0+gic_start_pmr_masking (); \ +arch_local_irq_restore (flags); \ + res; \This is in the critical path, so ideally this could instead be done inside the function, because the trap will set PSTATE.I already, so it should be possible to reduce this to a set of save/clear/restore operations after
__kvm_call_hyp.Quoted 4 lines
Hmmm, you are talking about the function called by
__kvm_call_hyp, right?But this means we'll need to add the save/clear/restore to each function that can be called by
__kvm_call_hyp, no?No, you only need to do this in the case where you run the guedt,
__kvm_vcpu_run, because all the other callers don't need to be able to take interrupts as they are under control of the host.Quoted 5 lines
Also in the VHE case we don't have the trap setting the PSTATE.I (the code calling
kvm_call_hypdisables interrupts before entering the guest, but now the function disabling interrupts is just masking them with PMR).
Quoted 5 lines
Yes, for VHE you'd have to do something more. One option would be to disable interrupts using PSTATE.I in
kvm_arch_vcpu_ioctl_run()for VHE, another option is to simply mask interrupts in the PSTATE only for VHE in__kvm_vcpu_run.
I agree this should work, but I wonder whether it is worth splitting the different places we have to deal with that.
Do you think there is good performance improvement to be had here?
Marc, what's your opinion on saving/restoring the PMR state in the hyp run code, relying on the I bit being always set before hand by hvc or explicitly when in VHE?
Quoted 38 lines
Quoted 38 lines
Quoted 37 lines
Quoted 23 lines
arch/arm64/include/asm/kvm_host.h +7
-0+ }) +#else #define kvm_call_hyp (f, ...) __kvm_call_hyp(kvm_ksym_ref(f), ##__VA_ARGS__) +#endif void force_vm_exit (const cpumask_t *mask);void kvm_mmu_wp_memory_region (struct kvm *kvm,int slot);@@ -349,6 +376,9 @@ static inline void __cpu_init_hyp_mode(phys_addr_t pgd_ptr,unsigned long hyp_stack_ptr, unsigned long vector_ptr) { +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + unsigned long flags; +#endif /* * Call initialization code, and switch to the full blown HYP code. * If the cpucaps haven't been finalized yet, something has gone very @@ -356,7 +386,17 @@ static inline void __cpu_init_hyp_mode(phys_addr_t pgd_ptr,* cpus_have_const_cap() wrapper. */ BUG_ON (!static_branch_likely (&arm64_const_caps_ready)); +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS arch/arm64/include/asm/kvm_host.h static inline void __cpu_init_hyp_mode(phys_addr_t pgd_ptr, · line 399
Quoted 10 lines
arch/arm64/include/asm/kvm_host.h +9
-0+ flags =arch_local_irq_save (); +gic_end_pmr_masking (); +#endif +__kvm_call_hyp ((void *)pgd_ptr, hyp_stack_ptr, vector_ptr); + +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +gic_start_pmr_masking (); +arch_local_irq_restore (flags); +#endif I don't think you need any of this, because I don't think you'd ever need to handle interrupts while initializing hyp mode.
Quoted 9 lines
Quoted 4 lines
I was more worried about getting an "NMI" during the EL2 initialisation rather than missing the masked interrupts. Do you know whether this might be an issue here or not?
For non-VHE it won't be an issue, because making a hyp call will mask interrupts using PSTATE.I, and for VHE we don't do any work here, so it's not an issue there either.
Noted, I'll get rid of that part.
Thanks,
Signature · 1 line
Julien Thierry
RFC 4/7 arm64: irqflags: Use ICC sysregs to implement IRQ masking
Quoted 152 lines · Julien Thierry
Quoted 143 lines · Christoffer Dall
Quoted 127 lines · Julien Thierry
Quoted 43 lines · Christoffer Dall
Quoted 35 lines · Julien Thierry
From: Daniel Thompson <[email protected]>
Currently irqflags is implemented using the PSR's I bit. It is possible to implement irqflags by using the co-processor interface to the GIC. Using the co-processor interface makes it feasible to simulate NMIs using GIC interrupt prioritization.
This patch changes the irqflags macros to modify, save and restore
ICC_PMR_EL1. This has a substantial knock on effect for the rest of the kernel. There are four reasons for this:1. The state of the PMR becomes part of the interrupt context and must be saved and restored during exceptions. It is saved on the stack as part of the saved context when an interrupt/exception is taken.
2. The hardware automatically masks the I bit (at boot, during traps, etc). When the I bit is set by hardware we must add code to switch from I bit masking and PMR masking:
- For IRQs, this is done after the interrupt has been acknowledged avoiding the need to unmask.
- For other exceptions, this is done right after saving the context.3. Some instructions, such as wfi, require that the PMR not be used for interrupt masking. Before calling these instructions we must switch from PMR masking to I bit masking.
This is also the case when KVM runs a guest, if the CPU receives an interrupt from the host, interrupts must not be masked in PMR otherwise the GIC will not signal it to the CPU.4. We use the alternatives system to allow a single kernel to boot and be switched to the alternative masking approach at runtime.
Signed-off-by Daniel Thompson <[email protected]>[julien.thierry at arm.com: changes reflected in commit,
message, fixes, renaming]Quoted 7 lines
Signed-off-by Julien Thierry <[email protected]>Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Christoffer Dall <[email protected]>Cc Marc Zyngier <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Jason Cooper <[email protected]>Quoted 6 lines
---
I just gave a quick look to the KVM part here but didn't get into whether this rather invasive change is warranted or not (it's definitely entertaining though).
I appreciate you looking into this, it wasn't very clear to me how to deal with that for KVM.
Quoted 21 lines
[...]
Quoted 18 lines
arch/arm64/include/asm/kvm_host.h +4
-0@@ -38,6 +38,10 @@#include <kvm/arm_arch_timer.h> #include <kvm/arm_pmu.h> +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +#include <asm/arch_gicv3.h> +#endif +#define KVM_MAX_VCPUS VGIC_V3_MAX_CPUS#define KVM_VCPU_MAX_FEATURES 4@@ -332,7 +336,30 @@ int kvm_unmap_hva_range(struct kvm *kvm,void kvm_arm_resume_guest (struct kvm *kvm);Quoted 11 lines
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 342
Quoted 5 lines
arch/arm64/include/asm/kvm_host.h +4
-0u64 __kvm_call_hyp (void *hypfn, ...); + +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +/* +* If we use PMR to disable interrupts, interrupts happening while the I would describe this as 'masking' interrupts, as opposed to 'disabling' interrupts, in general, and probably we can be more precise by categorizing interrupts as 'normal' vs. 'priority (NMIs)' or something like that.
Right, I'll do that.
Quoted 5 lines
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 343
Quoted 1 line
arch/arm64/include/asm/kvm_host.h +1
-0+* guest is executing will not be signaled to the host by the GIC (and signaled to the CPU. (Whether this then causes an exception to EL2 depends on the HCR_EL2.IMO flag and what KVM does with that exception).
True, I'll fix that.
Quoted 6 lines
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 345
Quoted 2 lines
arch/arm64/include/asm/kvm_host.h +2
-0+* any call to a waiting kvm_kick_many_cpus will likely cause a +* deadlock). This kick thing is an unccessary specific example to bring out here, I don't think it adds to the general understanding.
Yes, makes sense.
Quoted 10 lines
Quoted 10 lines
arch/arm64/include/asm/kvm_host.h +10
-0+* We need to switch back to using PSR.I. +*/ +#define kvm_call_hyp (f, ...) \ + ({ \ + u64 res; \ + unsigned long flags; \ + \ + flags = arch_local_irq_save(); \ + gic_end_pmr_masking(); \ + res = __kvm_call_hyp(kvm_ksym_ref(f), ##__VA_ARGS__); \Quoted 8 lines
arch/arm64/include/asm/kvm_host.h int kvm_unmap_hva_range(struct kvm *kvm, · line 358
Quoted 3 lines
arch/arm64/include/asm/kvm_host.h +3
-0+gic_start_pmr_masking (); \ +arch_local_irq_restore (flags); \ + res; \This is in the critical path, so ideally this could instead be done inside the function, because the trap will set PSTATE.I already, so it should be possible to reduce this to a set of save/clear/restore operations after
__kvm_call_hyp.Quoted 4 lines
Hmmm, you are talking about the function called by
__kvm_call_hyp, right?But this means we'll need to add the save/clear/restore to each function that can be called by
__kvm_call_hyp, no?No, you only need to do this in the case where you run the guedt,
__kvm_vcpu_run, because all the other callers don't need to be able to take interrupts as they are under control of the host.Quoted 5 lines
Also in the VHE case we don't have the trap setting the PSTATE.I (the code calling
kvm_call_hypdisables interrupts before entering the guest, but now the function disabling interrupts is just masking them with PMR).Quoted 5 lines
Yes, for VHE you'd have to do something more. One option would be to disable interrupts using PSTATE.I in
kvm_arch_vcpu_ioctl_run()for VHE, another option is to simply mask interrupts in the PSTATE only for VHE in__kvm_vcpu_run.
Quoted 8 lines
I agree this should work, but I wonder whether it is worth splitting the different places we have to deal with that.
Do you think there is good performance improvement to be had here?
Marc, what's your opinion on saving/restoring the PMR state in the hyp run code, relying on the I bit being always set before hand by hvc or explicitly when in VHE?
One thing I'm not overly keen on is to multiply the locations where we reconfigure the interrupt delivery, because it is overly fragile.
If I have followed what should happen, we have the following situation:
- pre-VHE: after HVC, unmask the PMR so that we can get interrupts when the guest is running. Restore it to its normal value before returning to the host
- VHE: Disable interrupts (mimicking the HVC), unmask PMR. Do the opposite on return.
At the moment, you do a bit too much, but that's because we only have a single entry point (__kvm_call_hyp). With Christoffer's VHE rework, we get a different entry point per architecture, probably making it simpler to hack something there.
You can have a look at those patches and see if that would make things easier...
Thanks,
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
RFC 5/7 irqchip/gic: Add functions to access irq priorities
In reply to Julien Thierry--- drivers/irqchip/irq-gic-common.c | 10 ++++++++++ drivers/irqchip/irq-gic-common.h | 2 ++ 2 files changed, 12 insertions(+)
drivers/irqchip/irq-gic-common.c +10 -0
@@ -90,6 +90,16 @@ int gic_configure_irq(unsigned int irq, unsigned int type,return ret; } +void gic_set_irq_prio (unsigned int irq,void __iomem *base,u8 prio) +{ +writeb_relaxed (prio, base +GIC_DIST_PRI + irq); +} + +u8 gic_get_irq_prio (unsigned int irq,void __iomem *base) +{ +return readb_relaxed (base +GIC_DIST_PRI + irq); +} +void gic_dist_config (void __iomem *base,int gic_irqs,void (*sync_access)(void )) {
drivers/irqchip/irq-gic-common.h +2 -0
@@ -35,6 +35,8 @@ void gic_dist_config(void __iomem *base, int gic_irqs,void gic_cpu_config (void __iomem *base,void (*sync_access)(void ));void gic_enable_quirks (u32 iidr,const struct gic_quirk *quirks,void *data); +void gic_set_irq_prio (unsigned int irq,void __iomem *base,u8 prio); +u8 gic_get_irq_prio (unsigned int irq,void __iomem *base);void gic_set_kvm_info (const struct gic_kvm_info *info);
--
1.9.1
RFC 6/7 arm64: Detect current view of GIC priorities
In reply to Julien ThierryThe values non secure EL1 needs to use for priority registers depends on the value of SCR_EL3.FIQ.
Since we don't have access to SCR_EL3, we fake an interrupt and compare the GIC priority with the one present in the [re]distributor.
Also, add firmware requirements related to SCR_EL3.
--- Documentation/arm64/booting.txt | 5 +++ arch/arm64/include/asm/arch_gicv3.h | 5 +++ arch/arm64/include/asm/irqflags.h | 6 +++ arch/arm64/include/asm/sysreg.h | 1 + drivers/irqchip/irq-gic-v3.c | 83 +++++++++++++++++++++++++++++++++++++ 5 files changed, 100 insertions(+)
Documentation/arm64/booting.txt +5 -0
@@ -188,6 +188,11 @@ Before jumping into the kernel, the following conditions must be met: the kernel image will be entered must be initialised by software at a higher exception level to prevent execution in an UNKNOWN state. + - SCR_EL3.FIQ must have the same value across all CPUs the kernel is + executing on. + - The value of SCR_EL3.FIQ must be the same as the one present at boot + time whenever the kernel is executing. + For systems with a GICv3 interrupt controller to be used in v3 mode: - If EL3 is present: ICC_SRE_EL3.Enable (bit 3) must be initialiased to 0b1.
arch/arm64/include/asm/arch_gicv3.h +5 -0
@@ -114,6 +114,11 @@ static inline void gic_write_bpr1(u32 val)write_sysreg_s (val,SYS_ICC_BPR1_EL1 ); } +static inline u32 gic_read_rpr (void ) +{ +return read_sysreg_s (SYS_ICC_RPR_EL1 ); +} +#define gic_read_typer (c) readq_relaxed(c)#define gic_write_irouter (v, c) writeq_relaxed(v, c)#define gic_read_lpir (c) readq_relaxed(c)
arch/arm64/include/asm/irqflags.h +6 -0
@@ -205,6 +205,12 @@ static inline int arch_irqs_disabled_flags(unsigned long flags) !(ARCH_FLAGS_GET_PMR(flags) & ICC_PMR_EL1_EN_BIT); } +/* Mask IRQs at CPU level instead of GIC level */ +static inline void arch_irqs_daif_disable (void ) +{ + asmvolatile ("msr daifset, #2" : : :"memory" ); +} +void maybe_switch_to_sysreg_gic_cpuif (void );#endif /* CONFIG_IRQFLAGS_GIC_MASKING */
arch/arm64/include/asm/sysreg.h +1 -0
@@ -205,6 +205,7 @@#define SYS_ICC_SRE_EL1 sys_reg(3, 0, 12, 12, 5)#define SYS_ICC_IGRPEN0_EL1 sys_reg(3, 0, 12, 12, 6)#define SYS_ICC_IGRPEN1_EL1 sys_reg(3, 0, 12, 12, 7) +#define SYS_ICC_RPR_EL1 sys_reg(3, 0, 12, 11, 3)#define SYS_CONTEXTIDR_EL1 sys_reg(3, 0, 13, 0, 1)#define SYS_TPIDR_EL1 sys_reg(3, 0, 13, 0, 4)
drivers/irqchip/irq-gic-v3.c +83 -0
101 lines not on this page · raw message
--
1.9.1
RFC 6/7 arm64: Detect current view of GIC priorities
Quoted 59 lines · Julien Thierry
The values non secure EL1 needs to use for priority registers depends on the value of SCR_EL3.FIQ.
Since we don't have access to
SCR_EL3, we fake an interrupt and compare the GIC priority with the one present in the [re]distributor.Also, add firmware requirements related to
SCR_EL3.Signed-off-by Julien Thierry <[email protected]>Cc Catalin Marinas <[email protected]>Cc Will Deacon <[email protected]>Cc Thomas Gleixner <[email protected]>Cc Jason Cooper <[email protected]>Cc Marc Zyngier <[email protected]>--- Documentation/arm64/booting.txt | 5 +++ arch/arm64/include/asm/arch_gicv3.h | 5 +++ arch/arm64/include/asm/irqflags.h | 6 +++ arch/arm64/include/asm/sysreg.h | 1 + drivers/irqchip/irq-gic-v3.c | 83 +++++++++++++++++++++++++++++++++++++ 5 files changed, 100 insertions(+)Documentation/arm64/booting.txt +5
-0@@ -188,6 +188,11 @@ Before jumping into the kernel, the following conditions must be met: the kernel image will be entered must be initialised by software at a higher exception level to prevent execution in an UNKNOWN state. + - SCR_EL3.FIQ must have the same value across all CPUs the kernel is + executing on. + - The value of SCR_EL3.FIQ must be the same as the one present at boot + time whenever the kernel is executing. + For systems with a GICv3 interrupt controller to be used in v3 mode: - If EL3 is present: ICC_SRE_EL3.Enable (bit 3) must be initialiased to 0b1.arch/arm64/include/asm/arch_gicv3.h +5
-0@@ -114,6 +114,11 @@ static inline void gic_write_bpr1(u32 val)write_sysreg_s (val,SYS_ICC_BPR1_EL1 ); } +static inline u32 gic_read_rpr (void ) +{ +return read_sysreg_s (SYS_ICC_RPR_EL1 ); +} +#define gic_read_typer (c) readq_relaxed(c)#define gic_write_irouter (v, c) writeq_relaxed(v, c)#define gic_read_lpir (c) readq_relaxed(c)arch/arm64/include/asm/irqflags.h +0
-0
arch/arm64/include/asm/irqflags.h static inline int arch_irqs_disabled_flags(unsigned long flags) · line 209
Quoted 5 lines
arch/arm64/include/asm/irqflags.h +2
-0@@ -205,6 +205,12 @@ static inline int arch_irqs_disabled_flags(unsigned long flags) !(ARCH_FLAGS_GET_PMR (flags) &ICC_PMR_EL1_EN_BIT ); } +/* Mask IRQs at CPU level instead of GIC level */ +static inline void arch_irqs_daif_disable (void )
nit: I find the naming pretty horrible. No, I don't have a better suggestion.
Quoted 12 lines
arch/arm64/include/asm/irqflags.h +4
-0+{ + asmvolatile ("msr daifset, #2" : : :"memory" ); +} +void maybe_switch_to_sysreg_gic_cpuif (void );#endif /* CONFIG_IRQFLAGS_GIC_MASKING */ arch/arm64/include/asm/sysreg.h +0
-0
arch/arm64/include/asm/sysreg.h line 208
Quoted 4 lines
arch/arm64/include/asm/sysreg.h +1
-0@@ -205,6 +205,7 @@#define SYS_ICC_SRE_EL1 sys_reg(3, 0, 12, 12, 5)#define SYS_ICC_IGRPEN0_EL1 sys_reg(3, 0, 12, 12, 6)#define SYS_ICC_IGRPEN1_EL1 sys_reg(3, 0, 12, 12, 7) +#define SYS_ICC_RPR_EL1 sys_reg(3, 0, 12, 11, 3)
We should try and keep the ordering of sysreg consistent, following the (Op0, Op1, CRn, CRm, Op2) encoding. And actually, this sysreg has already been added (see 43515894c06f8).
Quoted 62 lines
arch/arm64/include/asm/sysreg.h +0
-0#define SYS_CONTEXTIDR_EL1 sys_reg(3, 0, 13, 0, 1)#define SYS_TPIDR_EL1 sys_reg(3, 0, 13, 0, 4)drivers/irqchip/irq-gic-v3.c +44
-0@@ -62,6 +62,10 @@ struct gic_chip_data {static struct gic_chip_data gic_data__read_mostly ;static struct static_key supports_deactivate =STATIC_KEY_INIT_TRUE ; +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +DEFINE_STATIC_KEY_FALSE (have_non_secure_prio_view); +#endif +static struct gic_kvm_info gic_v3_kvm_info;#define gic_data_rdist () (this_cpu_ptr(gic_data.rdists.rdist))@@ -969,6 +973,81 @@ static int partition_domain_translate(struct irq_domain *d, .select = gic_irq_domain_select, }; +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +/* +* The behaviours of RPR and PMR registers differ depending on the value of +* SCR_EL3.FIQ, while the behaviour of priority registers of the distributor +* and redistributors is always the same. +* +* If SCR_EL3.FIQ == 1, the values used for RPR and PMR are the same as the ones +* programmed in the distributor and redistributors registers. +* +* Otherwise, the value presented by RPR as well as the value which will be +* compared against PMR is: (GIC_(R)DIST_PRI[irq] >> 1) | 0x80; +* +* see GICv3/GICv4 Architecture Specification (IHI0069D): +* - section 4.8.1 Non-secure accesses to register fields for Secure interrupt +* priorities. +* - Figure 4-7 Secure read of the priority field for a Non-secure Group 1 +* interrupt. +*/ +static void __init gic_detect_prio_view (void ) +{ +/* +* Randomly picked SGI, must be <= 8 as other SGIs might be +* used by the firmware. +*/ +const u32 fake_irqnr =7 ; +const u32 fake_irqmask =BIT (fake_irqnr); +void __iomem *const rdist_base =gic_data_rdist_sgi_base (); +unsigned long irq_flags; +u32 acked_irqnr; +bool was_enabled; + + irq_flags =arch_local_save_flags (); + +arch_irqs_daif_disable (); + + was_enabled = !!(readl_relaxed (rdist_base +GICD_ISENABLER ) & + fake_irqmask); + +if (!was_enabled) +writel_relaxed (fake_irqmask, rdist_base +GICD_ISENABLER );
drivers/irqchip/irq-gic-v3.c static int partition_domain_translate(struct irq_domain *d, · line 1025
Quoted 10 lines
drivers/irqchip/irq-gic-v3.c +10
-0+ +/* Need to unmask to acknowledge the IRQ */ +gic_write_pmr (ICC_PMR_EL1_UNMASKED ); +dsb (sy); + +/* Fake a pending SGI */ +writel_relaxed (fake_irqmask, rdist_base +GICD_ISPENDR ); +dsb (sy); + + acked_irqnr =gic_read_iar ();
Careful, there is no guarantee that you'll immediately get an interrupt here. There is a propagation delay, and the only thing the spec says is that you'll get something in finite time (give or take a few days).
drivers/irqchip/irq-gic-v3.c static int partition_domain_translate(struct irq_domain *d, · line 1038
Quoted 13 lines
drivers/irqchip/irq-gic-v3.c +13
-0+ +if (acked_irqnr == fake_irqnr) { +if (gic_read_rpr () ==gic_get_irq_prio (acked_irqnr, rdist_base)) +static_branch_enable (&have_non_secure_prio_view); + }else { +pr_warn ("Unexpected IRQ for priority detection: %u\n" , + acked_irqnr); + } + +if (acked_irqnr <1020 ) { +gic_write_eoir (acked_irqnr); +if (static_key_true (&supports_deactivate)) +gic_write_dir (acked_irqnr);
I think we could move this construct to a helper (it is otherwise used in the driver).
Quoted 27 lines
drivers/irqchip/irq-gic-v3.c +16
-1+ } + +/* Restore enabled state */ +if (!was_enabled) { +writel_relaxed (fake_irqmask, rdist_base +GICD_ICENABLER ); +gic_redist_wait_for_rwp (); + } + +arch_local_irq_restore (irq_flags); +} +#endif + static int __init gic_init_bases(void __iomem *dist_base, struct redist_region *rdist_regs, u32 nr_redist_regions,@@ -1025,6 +1104,10 @@ static int __init gic_init_bases(void __iomem *dist_base,gic_cpu_init ();gic_cpu_pm_init (); +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +gic_detect_prio_view (); +#endif +return 0 ; out_free: --
Quoted 2 lines
1.9.1
Thanks,
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
RFC 7/7 arm64: Add support for pseudo-NMIs
In reply to Julien Thierryarm64 does not provide native NMIs. Emulate the NMI behaviour using GIC priorities.
Add the possibility to set an IRQ as an NMI and the handling of the NMI.
If the view of GIC priorities is the secure one (i.e. SCR_EL3.FIQ == 0), do not allow the use of NMIs. Emit a warning when attempting to set an IRQ as NMI under this scenario.
--- arch/arm64/kernel/entry.S | 54 +++++++++++++++++ drivers/irqchip/irq-gic-v3.c | 141 +++++++++++++++++++++++++++++++++++++++++++ include/linux/interrupt.h | 1 + 3 files changed, 196 insertions(+)
arch/arm64/kernel/entry.S +54 -0
@@ -365,6 +365,16 @@ alternative_else_nop_endif msr daifclr, #2 alternative_else_nop_endif .endm + + /* Should be checked on return from irq handlers */ + .macro branch_if_was_nmi, tmp, target + alternative_if ARM64_HAS_SYSREG_GIC_CPUIF + mrs \tmp, daif + alternative_else + mov \tmp, #0 + alternative_endif + tbnz \tmp, #7, \target // Exiting an NMI + .endm #endif /*
@@ -610,12 +620,30 @@ ENDPROC(el1_sync) el1_irq: kernel_entry 1 enable_dbg + #ifdef CONFIG_TRACE_IRQFLAGS +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + ldr x20, [sp, #S_PMR_SAVE] + /* Irqs were disabled, don't trace */ + tbz x20, ICC_PMR_EL1_EN_SHIFT, 1f +#endif bl trace_hardirqs_off +1: #endif irq_handler +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + /* + * Irqs were disabled, we have an nmi. + * We might have interrupted a context with interrupt disabled that set + * NEED_RESCHED flag. + * Skip preemption and irq tracing if needed. + */ + tbz x20, ICC_PMR_EL1_EN_SHIFT, untraced_irq_exit + branch_if_was_nmi x0, skip_preempt +#endif + #ifdef CONFIG_PREEMPT ldr w24, [tsk, #TSK_TI_PREEMPT] // get preempt count cbnz w24, 1f // preempt count != 0
@@ -624,9 +652,13 @@ el1_irq: bl el1_preempt 1: #endif + +skip_preempt: #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_on #endif + +untraced_irq_exit: kernel_exit 1 ENDPROC(el1_irq)
@@ -839,6 +871,11 @@ el0_irq_naked: #ifdef CONFIG_TRACE_IRQFLAGS bl trace_hardirqs_on #endif + +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + branch_if_was_nmi x2, nmi_ret_to_user +#endif + b ret_to_user ENDPROC(el0_irq)
@@ -984,8 +1021,15 @@ ENTRY(cpu_switch_to) ldp x27, x28, [x8], #16 ldp x29, x9, [x8], #16 ldr lr, [x8] +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + mrs x10, daif + msr daifset, #2 +#endif mov sp, x9 msr sp_el0, x1 +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS + msr daif, x10 +#endif ret ENDPROC(cpu_switch_to) NOKPROBE(cpu_switch_to)
@@ -1002,3 +1046,13 @@ ENTRY(ret_from_fork) b ret_to_user ENDPROC(ret_from_fork) NOKPROBE(ret_from_fork) + +#ifdef CONFIG_USE_ICC_SYSREGS_FOR_IRQFLAGS +/* + * NMI return path to EL0 + */ +nmi_ret_to_user: + ldr x1, [tsk, #TSK_TI_FLAGS] + b finish_ret_to_user +ENDPROC(nmi_ret_to_user) +#endif
drivers/irqchip/irq-gic-v3.c +141 -0
189 lines not on this page · raw message
include/linux/interrupt.h +1 -0
@@ -420,6 +420,7 @@ enum irqchip_irq_state {IRQCHIP_STATE_ACTIVE ,/* Is interrupt in progress? */ IRQCHIP_STATE_MASKED ,/* Is interrupt masked? */ IRQCHIP_STATE_LINE_LEVEL ,/* Is IRQ line high? */ + IRQCHIP_STATE_NMI, /* Is IRQ an NMI? */ }; extern int irq_get_irqchip_state(unsigned int irq, enum irqchip_irq_state which,
--
1.9.1
cover arm64: provide pseudo NMI with GICv3
In reply to Julien ThierryQuoted 4 lines · Julien Thierry
Hi,
This series is a continuation of the work started by Daniel [1]. The goal is to use GICv3 interrupt priorities to simulate an NMI.
Cool. By coincidence, and after ignoring the code for three kernel releases, I'm halfway through rebasing this to v4.14... and *very* happy to be able to grab yours instead. There's a machine I want to play with it on!
Have you just brought it up to date or have you acted on Marc Z.'s pending feedback from the last circulation?
Quoted 9 lines
To achieve this, set two priorities, one for standard interrupts and another, higher priority, for NMIs. Whenever we want to disable interrupts, we mask the standard priority instead so NMIs can still be raised. Some corner cases though still require to actually mask all interrupts effectively disabling the NMI.
Of course, using priority masking instead of PSR.I comes at some cost. On hackbench, I notice a ~6% drop of performance. However, on a kernel build, I do not see significant differences.
Which micro-architecture?
In micro-benchmarks (with hot caches) I've seen the lock sequence speed up on some designs, slow down a bit on some and slow down a *lot* others. In the worst cases the code impacts Android macro benchmarks like scrolling... on the other hand the code was stable enough to run the whole Android stack reliably.
Daniel.
cover arm64: provide pseudo NMI with GICv3
Hi Daniel,
Quoted 11 lines · Daniel Thompson
Quoted 4 lines · Julien Thierry
Hi,
This series is a continuation of the work started by Daniel [1]. The goal is to use GICv3 interrupt priorities to simulate an NMI.
Cool. By coincidence, and after ignoring the code for three kernel releases, I'm halfway through rebasing this to v4.14... and *very* happy to be able to grab yours instead. There's a machine I want to play with it on!
Good to know someone will be testing this then ;) , thanks.
Quoted 3 lines
Have you just brought it up to date or have you acted on Marc Z.'s pending feedback from the last circulation?
First 3 patches should be pretty much identical. I've done a bit of rework on patch 4, acting on some of Marc's comments, fixing some issues I found along the road, some refactoring.
Then I added code to the GICv3 irqchip so you can turn an irq as NMI like so:irq_set_irqchip_state(virq, IRQCHIP_STATE_NMI, true);
So this means you need to do it through a virtual irq number (e.g. no IPI). I started working on some patches to make it work with IPI as well, but I wanted to get some feeback on this first before finalizing them.
Quoted 7 lines
Quoted 7 lines
To achieve this, set two priorities, one for standard interrupts and another, higher priority, for NMIs. Whenever we want to disable interrupts, we mask the standard priority instead so NMIs can still be raised. Some corner cases though still require to actually mask all interrupts effectively disabling the NMI.
Quoted 6 lines
Quoted 3 lines
Of course, using priority masking instead of PSR.I comes at some cost. On hackbench, I notice a ~6% drop of performance. However, on a kernel build, I do not see significant differences.
Which micro-architecture?
8 cores A57.
Quoted 6 lines
In micro-benchmarks (with hot caches) I've seen the lock sequence speed up on some designs, slow down a bit on some and slow down a *lot* others. In the worst cases the code impacts Android macro benchmarks like scrolling... on the other hand the code was stable enough to run the whole Android stack reliably.
So far I only tested with classic Linux distros, so if you find issues with Android (or other) do let me know.
Same if you have any question about the changes.
Thanks,
Signature · 1 line
Julien Thierry
cover arm64: provide pseudo NMI with GICv3
In reply to Julien ThierryGentle ping on this series.
Quoted 88 lines · Julien Thierry
Hi,
This series is a continuation of the work started by Daniel [1]. The goal is to use GICv3 interrupt priorities to simulate an NMI.
To achieve this, set two priorities, one for standard interrupts and another, higher priority, for NMIs. Whenever we want to disable interrupts, we mask the standard priority instead so NMIs can still be raised. Some corner cases though still require to actually mask all interrupts effectively disabling the NMI.
Of course, using priority masking instead of PSR.I comes at some cost. On hackbench, I notice a ~6% drop of performance. However, on a kernel build, I do not see significant differences.
Currently, only PPIs and SPIs can be set as NMIs. IPIs being currently hardcoded IRQ numbers, there isn't a generic interface to set SGIs as NMI for now. I don't think there is any reason LPIs should be allowed to be set as NMI as they do not have an active state.
When an NMI is active on a CPU, no other NMI can be triggered on the CPU.Requirements to use this:
- Have GICv3
- SCR_EL3.FIQ is set to 1 when linux runs
- Select Kernel Feature -> Use ICC system registers for IRQ maskingFrom my testing with the perf interrupt, there doesn't seem to be any major
issue left (but if you find some, do tell!).* Patches 1 and 5 are minor utility changes.
* Patches 2 and 3 allows to detect and enable the use of GICv3 system registers during boot time.
* Patch 4 introduces the masking of IRQs using priorities replacing irq disabling.
* Patch 6 add detection of the view linux has on GICv3 priorities, without this we cannot easily mask specific priorities in an accurate manner
* Patch 7 adds the support for NMIs[1] http://www.spinics.net/lists/arm-kernel/msg525077.html
Thanks,
Julien Thierry
Daniel Thompson (4):
arm64: irqflags: Reorder the fiq & async macros
arm64: cpufeature: Allow early detect of specific features
arm64: alternative: Apply alternatives early in boot process
arm64: irqflags: Use ICC sysregs to implement IRQ maskingJulien Thierry (3):
irqchip/gic: Add functions to access irq priorities
arm64: Detect current view of GIC priorities
arm64: Add support for pseudo-NMIsDocumentation/arm64/booting.txt | 5 + arch/arm64/Kconfig | 15 ++ arch/arm64/include/asm/alternative.h | 1 + arch/arm64/include/asm/arch_gicv3.h | 24 +++ arch/arm64/include/asm/assembler.h | 33 +++- arch/arm64/include/asm/efi.h | 5 + arch/arm64/include/asm/irqflags.h | 143 +++++++++++++++- arch/arm64/include/asm/kvm_host.h | 40 +++++ arch/arm64/include/asm/processor.h | 4 + arch/arm64/include/asm/ptrace.h | 14 +- arch/arm64/include/asm/sysreg.h | 1 + arch/arm64/kernel/alternative.c | 39 ++++- arch/arm64/kernel/asm-offsets.c | 1 + arch/arm64/kernel/cpufeature.c | 67 +++++--- arch/arm64/kernel/entry.S | 132 ++++++++++++-- arch/arm64/kernel/head.S | 38 +++++ arch/arm64/kernel/process.c | 6 + arch/arm64/kernel/smp.c | 14 ++ arch/arm64/mm/proc.S | 23 +++ drivers/irqchip/irq-gic-common.c | 10 ++ drivers/irqchip/irq-gic-common.h | 2 + drivers/irqchip/irq-gic-v3-its.c | 2 +- drivers/irqchip/irq-gic-v3.c | 304 +++++++++++++++++++++++++++++---- include/linux/interrupt.h | 1 + include/linux/irqchip/arm-gic-common.h | 6 + include/linux/irqchip/arm-gic.h | 5 - 26 files changed, 845 insertions(+), 90 deletions(-)--
1.9.1
Signature · 1 line
Julien Thierry