Split off nVHE hyp code
v3 cover Split off nVHE hyp code
Refactor files in arch/arm64/kvm/hyp to compile all code which runs in EL2 under nVHE into separate object files from the rest of KVM. This is done in preparation for being able to unmap hyp code from EL1 and kernel code/data from EL2 but has other benefits too, notably:
* safe use of KASAN/UBSAN/GCOV instrumentation on VHE code,
* no need for __hyp_text annotations.
nVHE-specific code is moved to hyp/nvhe and compiled with custom build rules similar to those used by EFI stub. Shared source files are compiled under both VHE and nVHE build rules. Where a source file contained both VHE and nVHE code, it is split into a shared header file and two C source files. This is done one file per commit to make review easier.
All nVHE symbols are prefixed with "__kvm_nvhe_" to avoid collisions with VHE variants (also inspired by EFI stub). Since this prefixes unresolved symbols too, image-vars.h contains a list of kernel symbol aliases where nVHE code still refers to kernel proper. The list grows fairly large as the patch series progresses and code is moved around, but at the end contains 20 symbols. These remaining dependencies on kernel proper will be further reduced in the future.
No functional changes are intended but code was simplified whenever the refactoring made it possible.
Tested by running kvm-unit-tests on QEMU 5.0 with VHE/nVHE and GIC v2/v3.
Dual compilation of code shared by VHE/nVHE increase the size of the kernel. Bloat-o-meter vmlinux diff shows an increase of 21 KB on the ELF symbol level. Size of Image.gz is up by 10 KB; size of Image is unchanged, presumably due to ELF section alignment.
This is based off v5.8-rc1. Available in branch 'topic/el2-obj-v3' of git repo:
https://android-kvm.googlesource.com/linux
Changes v2 -> v3:
* rebase onto v5.8-rc1
* remove patch changing hypcall interface to function IDs
* move hyp-init.S to nVHE
* fix symbol aliasing under CONFIG_ARM64_PSEUDO_NMI and CONFIG_ARM64_SVE
* remove VHE's unused __kvm_enable_ssbs()
* make nVHE use VHE's hyp_panic_string for consistent use of absolute relocs when returning pointers back to kernel
* replace __noscs annotation (added to __hyp_text macro) with build rule
Changes v1 -> v2:
* change nVHE symbol prefix from __hyp_text_ to __kvm_nvhe_
* rename __HYPERVISOR__ macro to __KVM_NVHE_HYPERVISOR__
* use hcall jump table instead of array of function pointers
* drop patch to unify HVC callers
* move __smccc_workaround_1_smc to own file
* header guards for hyp/*.h
* improve helpers for handling VHE/nVHE hyp syms in kernel proper
* improve commit messages, cover letter
-David
Andrew Scull (2):
arm64: kvm: Handle calls to prefixed hyp functions
arm64: kvm: Move hyp-init.S to nVHE
David Brazdil (13):
arm64: kvm: Fix symbol dependency in __hyp_call_panic_nvhe
arm64: kvm: Move __smccc_workaround_1_smc to .rodata
arm64: kvm: Add build rules for separate nVHE object files
arm64: kvm: Build hyp-entry.S separately for VHE/nVHE
arm64: kvm: Split hyp/tlb.c to VHE/nVHE
arm64: kvm: Split hyp/switch.c to VHE/nVHE
arm64: kvm: Split hyp/debug-sr.c to VHE/nVHE
arm64: kvm: Split hyp/sysreg-sr.c to VHE/nVHE
arm64: kvm: Split hyp/timer-sr.c to VHE/nVHE
arm64: kvm: Compile remaining hyp/ files for both VHE/nVHE
arm64: kvm: Add comments around __kvm_nvhe_ symbol aliases
arm64: kvm: Remove __hyp_text macro, use build rules instead
arm64: kvm: Lift instrumentation restrictions on VHE
arch/arm64/include/asm/kvm_asm.h | 32 +-
arch/arm64/include/asm/kvm_emulate.h | 2 +-
arch/arm64/include/asm/kvm_host.h | 19 +-
arch/arm64/include/asm/kvm_hyp.h | 13 +-
arch/arm64/include/asm/kvm_mmu.h | 16 +-
arch/arm64/include/asm/mmu.h | 7 -
arch/arm64/kernel/cpu_errata.c | 4 +-
arch/arm64/kernel/image-vars.h | 50 ++
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/arm.c | 8 +-
arch/arm64/kvm/hyp/Makefile | 11 +-
arch/arm64/kvm/hyp/aarch32.c | 6 +-
arch/arm64/kvm/hyp/debug-sr.c | 210 +------
arch/arm64/kvm/hyp/debug-sr.h | 170 +++++
arch/arm64/kvm/hyp/entry.S | 1 -
arch/arm64/kvm/hyp/fpsimd.S | 1 -
arch/arm64/kvm/hyp/hyp-entry.S | 21 +-
arch/arm64/kvm/hyp/nvhe/Makefile | 43 ++
arch/arm64/kvm/hyp/nvhe/debug-sr.c | 77 +++
arch/arm64/kvm/{ => hyp/nvhe}/hyp-init.S | 0
arch/arm64/kvm/hyp/nvhe/switch.c | 271 ++++++++
arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 56 ++
arch/arm64/kvm/hyp/nvhe/timer-sr.c | 43 ++
arch/arm64/kvm/hyp/nvhe/tlb.c | 68 ++
arch/arm64/kvm/hyp/smccc_wa.S | 30 +
arch/arm64/kvm/hyp/switch.c | 749 +----------------------
arch/arm64/kvm/hyp/switch.h | 504 +++++++++++++++
arch/arm64/kvm/hyp/sysreg-sr.c | 215 +------
arch/arm64/kvm/hyp/sysreg-sr.h | 204 ++++++
arch/arm64/kvm/hyp/timer-sr.c | 38 +-
arch/arm64/kvm/hyp/tlb.c | 169 +----
arch/arm64/kvm/hyp/tlb.h | 131 ++++
arch/arm64/kvm/hyp/vgic-v2-cpuif-proxy.c | 4 +-
arch/arm64/kvm/hyp/vgic-v3-sr.c | 130 ++--
arch/arm64/kvm/mmu.c | 2 +-
arch/arm64/kvm/va_layout.c | 2 +-
scripts/kallsyms.c | 1 +
37 files changed, 1829 insertions(+), 1481 deletions(-)
create mode 100644 arch/arm64/kvm/hyp/debug-sr.h
create mode 100644 arch/arm64/kvm/hyp/nvhe/Makefile
create mode 100644 arch/arm64/kvm/hyp/nvhe/debug-sr.c
rename arch/arm64/kvm/{ => hyp/nvhe}/hyp-init.S (100%)
create mode 100644 arch/arm64/kvm/hyp/nvhe/switch.c
create mode 100644 arch/arm64/kvm/hyp/nvhe/sysreg-sr.c
create mode 100644 arch/arm64/kvm/hyp/nvhe/timer-sr.c
create mode 100644 arch/arm64/kvm/hyp/nvhe/tlb.c
create mode 100644 arch/arm64/kvm/hyp/smccc_wa.S
create mode 100644 arch/arm64/kvm/hyp/switch.h
create mode 100644 arch/arm64/kvm/hyp/sysreg-sr.h
create mode 100644 arch/arm64/kvm/hyp/tlb.hSignature · 6 lines
2.27.0 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v3 1/15 arm64: kvm: Fix symbol dependency in __hyp_call_panic_nvhe
__hyp_call_panic_nvhe contains inline assembly which did not declare its dependency on the __hyp_panic_string symbol.
The static-declared string has previously been kept alive because of a use in __hyp_call_panic_vhe. Fix this in preparation for separating the source files between VHE and nVHE when the two users land in two different compilation units. The static variable otherwise gets dropped when compiling the nVHE source file, causing an undefined symbol linker error later.
--- arch/arm64/kvm/hyp/switch.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/kvm/hyp/switch.c +1 -1
@@ -897,7 +897,7 @@ static void __hyp_text __hyp_call_panic_nvhe(u64 spsr, u64 elr, u64 par,* making sure it is a kernel address and not a PC-relative * reference. */ - asmvolatile ("ldr %0, =__hyp_panic_string" :"=r" (str_va)); + asmvolatile ("ldr %0, =%1" :"=r" (str_va) :"S" (__hyp_panic_string)); __hyp_do_panic(str_va, spsr, elr,
Signature · 6 lines
2.27.0 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v3 2/15 arm64: kvm: Move __smccc_workaround_1_smc to .rodata
In reply to David BrazdilThis snippet of assembly is used by cpu_errata.c to overwrite parts of KVM hyp vector. Move it to its own source file and change its ELF section to .rodata.
--- arch/arm64/kvm/hyp/Makefile | 1 + arch/arm64/kvm/hyp/hyp-entry.S | 16 ---------------- arch/arm64/kvm/hyp/smccc_wa.S | 30 ++++++++++++++++++++++++++++++ 3 files changed, 31 insertions(+), 16 deletions(-) create mode 100644 arch/arm64/kvm/hyp/smccc_wa.S
arch/arm64/kvm/hyp/Makefile +1 -0
@@ -7,6 +7,7 @@ ccflags-y += -fno-stack-protector -DDISABLE_BRANCH_PROFILING \ $(DISABLE_STACKLEAK_PLUGIN) obj-$(CONFIG_KVM) += hyp.o +obj-$(CONFIG_KVM_INDIRECT_VECTORS) += smccc_wa.o hyp-y := vgic-v3-sr.o timer-sr.o aarch32.o vgic-v2-cpuif-proxy.o sysreg-sr.o \ debug-sr.o entry.o switch.o fpsimd.o tlb.o hyp-entry.o
arch/arm64/kvm/hyp/hyp-entry.S +0 -16
@@ -318,20 +318,4 @@ SYM_CODE_START(__bp_harden_hyp_vecs) 1: .org __bp_harden_hyp_vecs + __BP_HARDEN_HYP_VECS_SZ .org 1b SYM_CODE_END(__bp_harden_hyp_vecs) - - .popsection - -SYM_CODE_START(__smccc_workaround_1_smc) - esb - sub sp, sp, #(8 * 4) - stp x2, x3, [sp, #(8 * 0)] - stp x0, x1, [sp, #(8 * 2)] - mov w0, #ARM_SMCCC_ARCH_WORKAROUND_1 - smc #0 - ldp x2, x3, [sp, #(8 * 0)] - ldp x0, x1, [sp, #(8 * 2)] - add sp, sp, #(8 * 4) -1: .org __smccc_workaround_1_smc + __SMCCC_WORKAROUND_1_SMC_SZ - .org 1b -SYM_CODE_END(__smccc_workaround_1_smc) #endif
arch/arm64/kvm/hyp/smccc_wa.S new file +30 -0
@@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2015-2018 - ARM Ltd + * Author: Marc Zyngier <[email protected]> + */ + +#include <linux/arm-smccc.h> + +#include <asm/kvm_asm.h> +#include <asm/kvm_mmu.h> + + /* + * This is not executed directly and is instead copied into the vectors + * by install_bp_hardening_cb(). + */ + .data + .pushsection .rodata + .global __smccc_workaround_1_smc +__smccc_workaround_1_smc: + esb + sub sp, sp, #(8 * 4) + stp x2, x3, [sp, #(8 * 0)] + stp x0, x1, [sp, #(8 * 2)] + mov w0, #ARM_SMCCC_ARCH_WORKAROUND_1 + smc #0 + ldp x2, x3, [sp, #(8 * 0)] + ldp x0, x1, [sp, #(8 * 2)] + add sp, sp, #(8 * 4) +1: .org __smccc_workaround_1_smc + __SMCCC_WORKAROUND_1_SMC_SZ + .org 1b
Signature · 6 lines
2.27.0 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Marc Zyngier
maintainerRe: [PATCH v3 02/15] arm64: kvm: Move __smccc_workaround_1_smc to .rodata
Hi David,
Quoted 30 lines · David Brazdil
This snippet of assembly is used by
cpu_errata.cto overwrite parts of KVM hyp vector. Move it to its own source file and change its ELF section to .rodata.Signed-off-by David Brazdil <[email protected]>--- arch/arm64/kvm/hyp/Makefile | 1 + arch/arm64/kvm/hyp/hyp-entry.S | 16 ---------------- arch/arm64/kvm/hyp/smccc_wa.S | 30 ++++++++++++++++++++++++++++++ 3 files changed, 31 insertions(+), 16 deletions(-) create mode 100644 arch/arm64/kvm/hyp/smccc_wa.Sarch/arm64/kvm/hyp/Makefile +1
-1@@ -7,6 +7,7 @@ ccflags-y += -fno-stack-protector -DDISABLE_BRANCH_PROFILING \ $(DISABLE_STACKLEAK_PLUGIN) obj-$(CONFIG_KVM) += hyp.o +obj-$(CONFIG_KVM_INDIRECT_VECTORS) += smccc_wa.o hyp-y := vgic-v3-sr.o timer-sr.o aarch32.o vgic-v2-cpuif-proxy.osysreg-sr.o \
arch/arm64/kvm/hyp/Makefile +0
-0debug-sr.o entry.o switch.o fpsimd.o tlb.o hyp-entry.oarch/arm64/kvm/hyp/hyp-entry.S +0
-0
arch/arm64/kvm/hyp/hyp-entry.S SYM_CODE_START(__bp_harden_hyp_vecs) · line 322
Quoted 5 lines
arch/arm64/kvm/hyp/hyp-entry.S +0
-2@@ -318,20 +318,4 @@ SYM_CODE_START(__bp_harden_hyp_vecs) 1: .org __bp_harden_hyp_vecs + __BP_HARDEN_HYP_VECS_SZ .org 1b SYM_CODE_END(__bp_harden_hyp_vecs) - - .popsection
I'd be tempted to leave the .popsection in place, if only for symmetry with the initial .pushsection.
Quoted 35 lines
arch/arm64/kvm/hyp/hyp-entry.S +0
-14- -SYM_CODE_START(__smccc_workaround_1_smc) - esb - sub sp, sp, #(8 * 4) - stp x2, x3, [sp, #(8 * 0)] - stp x0, x1, [sp, #(8 * 2)] - mov w0, #ARM_SMCCC_ARCH_WORKAROUND_1 - smc #0 - ldp x2, x3, [sp, #(8 * 0)] - ldp x0, x1, [sp, #(8 * 2)] - add sp, sp, #(8 * 4) -1: .org __smccc_workaround_1_smc + __SMCCC_WORKAROUND_1_SMC_SZ - .org 1b -SYM_CODE_END(__smccc_workaround_1_smc) #endifarch/arm64/kvm/hyp/smccc_wa.S new file +13
-0@@ -0,0 +1,30 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (C) 2015-2018 - ARM Ltd + * Author: Marc Zyngier <[email protected]> + */ + +#include <linux/arm-smccc.h> + +#include <asm/kvm_asm.h> +#include <asm/kvm_mmu.h> + + /* + * This is not executed directly and is instead copied into thevectors
arch/arm64/kvm/hyp/smccc_wa.S line 19
Quoted 6 lines
arch/arm64/kvm/hyp/smccc_wa.S new file +6
-0+ * by install_bp_hardening_cb(). + */ + .data + .pushsection .rodata + .global __smccc_workaround_1_smc +__smccc_workaround_1_smc:
You probably want to replace this with SYM_DATA_START (and SYM_DATA_END at the end).
arch/arm64/kvm/hyp/smccc_wa.S line 30
Quoted 11 lines
arch/arm64/kvm/hyp/smccc_wa.S new file +11
-0+ esb + sub sp, sp, #(8 * 4) + stp x2, x3, [sp, #(8 * 0)] + stp x0, x1, [sp, #(8 * 2)] + mov w0, #ARM_SMCCC_ARCH_WORKAROUND_1 + smc #0 + ldp x2, x3, [sp, #(8 * 0)] + ldp x0, x1, [sp, #(8 * 2)] + add sp, sp, #(8 * 4) +1: .org __smccc_workaround_1_smc + __SMCCC_WORKAROUND_1_SMC_SZ + .org 1b
Otherwise, looks good.
Thanks,
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
Re: [PATCH v3 02/15] arm64: kvm: Move __smccc_workaround_1_smc to .rodata
Hey Marc,
Quoted 4 lines
arch/arm64/kvm/hyp/hyp-entry.S SYM_CODE_START(__bp_harden_hyp_vecs) · line 322
Quoted 1 line
arch/arm64/kvm/hyp/hyp-entry.S +0
-1- .popsectionI'd be tempted to leave the .popsection in place, if only for symmetry with the initial .pushsection.
I removed it because other .S files don't pop either. It must have been added here purely for the smccc workaround code. Happy to add it back if you prefer, but the pushsection is removed later in the series, so this would disappear as well.
Quoted 6 lines
arch/arm64/kvm/hyp/smccc_wa.S line 19
Quoted 3 lines
arch/arm64/kvm/hyp/smccc_wa.S new file +3
-0+ .pushsection .rodata + .global __smccc_workaround_1_smc +__smccc_workaround_1_smc:You probably want to replace this with
SYM_DATA_START(andSYM_DATA_ENDat the end).
Done
Thanks for reviewing,
David
Marc Zyngier
maintainerRe: [PATCH v3 02/15] arm64: kvm: Move __smccc_workaround_1_smc to .rodata
Quoted 8 lines · David Brazdil
Hey Marc,
Quoted 5 lines
arch/arm64/kvm/hyp/hyp-entry.S SYM_CODE_START(__bp_harden_hyp_vecs) · line 322
Quoted 1 line
arch/arm64/kvm/hyp/hyp-entry.S +0
-1- .popsectionI'd be tempted to leave the .popsection in place, if only for symmetry
with
the initial .pushsection.
Quoted 4 lines
I removed it because other .S files don't pop either. It must have been added here purely for the smccc workaround code. Happy to add it back if you prefer, but the pushsection is removed later in the series, so this would disappear as well.
Don't bother then.
Quoted 9 lines
Quoted 7 lines
arch/arm64/kvm/hyp/smccc_wa.S line 19
Quoted 4 lines
arch/arm64/kvm/hyp/smccc_wa.S new file +3
-0+ .pushsection .rodata + .global __smccc_workaround_1_smc +__smccc_workaround_1_smc:You probably want to replace this with
SYM_DATA_START(andSYM_DATA_ENDat the end).Done
Thanks!
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
PATCH v3 3/15 arm64: kvm: Add build rules for separate nVHE object files
In reply to David BrazdilAdd new folder arch/arm64/kvm/hyp/nvhe and a Makefile for building code that runs in EL2 under nVHE KVM.
Compile each source file into a .hyp.tmp.o object first, then prefix all its symbols with "__kvm_nvhe_" using objcopy and produce a .hyp.o. Suffixes were chosen so that it would be possible for VHE and nVHE to share some source files, but compiled with different CFLAGS. nVHE build rules add -D__KVM_NVHE_HYPERVISOR__.
The nVHE ELF symbol prefix is added to kallsyms.c as ignored. EL2-only symbols will never appear in EL1 stack traces.
--- arch/arm64/kernel/image-vars.h | 12 +++++++++++ arch/arm64/kvm/hyp/Makefile | 2 +- arch/arm64/kvm/hyp/nvhe/Makefile | 35 ++++++++++++++++++++++++++++++++ scripts/kallsyms.c | 1 + 4 files changed, 49 insertions(+), 1 deletion(-) create mode 100644 arch/arm64/kvm/hyp/nvhe/Makefile
arch/arm64/kernel/image-vars.h +12 -0
@@ -51,4 +51,16 @@ __efistub__ctype = _ctype;#endif +#ifdef CONFIG_KVM + +/* +* KVM nVHE code has its own symbol namespace prefixed by __kvm_nvhe_, to +* isolate it from the kernel proper. The following symbols are legally +* accessed by it, therefore provide aliases to make them linkable. +* Do not include symbols which may not be safely accessed under hypervisor +* memory mappings. +*/ + +#endif /* CONFIG_KVM */ +#endif /* __ARM64_KERNEL_IMAGE_VARS_H */
arch/arm64/kvm/hyp/Makefile +1 -1
@@ -6,7 +6,7 @@ ccflags-y += -fno-stack-protector -DDISABLE_BRANCH_PROFILING \ $(DISABLE_STACKLEAK_PLUGIN) -obj-$(CONFIG_KVM) += hyp.o +obj-$(CONFIG_KVM) += hyp.o nvhe/ obj-$(CONFIG_KVM_INDIRECT_VECTORS) += smccc_wa.o hyp-y := vgic-v3-sr.o timer-sr.o aarch32.o vgic-v2-cpuif-proxy.o sysreg-sr.o \
arch/arm64/kvm/hyp/nvhe/Makefile new file +35 -0
@@ -0,0 +1,35 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Makefile for Kernel-based Virtual Machine module, HYP/nVHE part +# + +asflags-y := -D__KVM_NVHE_HYPERVISOR__ +ccflags-y := -D__KVM_NVHE_HYPERVISOR__ -fno-stack-protector \ + -DDISABLE_BRANCH_PROFILING $(DISABLE_STACKLEAK_PLUGIN) + +obj-y := + +obj-y := $(patsubst %.o,%.hyp.o,$(obj-y)) +extra-y := $(patsubst %.hyp.o,%.hyp.tmp.o,$(obj-y)) + +$(obj)/%.hyp.tmp.o: $(src)/%.c FORCE + $(call if_changed_rule,cc_o_c) +$(obj)/%.hyp.tmp.o: $(src)/%.S FORCE + $(call if_changed_rule,as_o_S) +$(obj)/%.hyp.o: $(obj)/%.hyp.tmp.o FORCE + $(call if_changed,hypcopy) + +quiet_cmd_hypcopy = HYPCOPY $@ + cmd_hypcopy = $(OBJCOPY) --prefix-symbols=__kvm_nvhe_ $< $@ + +# KVM nVHE code is run at a different exception code with a different map, so +# compiler instrumentation that inserts callbacks or checks into the code may +# cause crashes. Just disable it. +GCOV_PROFILE := n +KASAN_SANITIZE := n +UBSAN_SANITIZE := n +KCOV_INSTRUMENT := n + +# Skip objtool checking for this directory because nVHE code is compiled with +# non-standard build rules. +OBJECT_FILES_NON_STANDARD := y
scripts/kallsyms.c +1 -0
@@ -109,6 +109,7 @@ static bool is_ignored_symbol(const char *name, char type)".LASANPC" ,/* s390 kasan local symbols */ "__crc_" ,/* modversions */ "__efistub_" ,/* arm64 EFI stub namespace */ +"__kvm_nvhe_" ,/* arm64 non-VHE KVM namespace */ NULL };
Signature · 6 lines
2.27.0 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Marc Zyngier
maintainerRe: [PATCH v3 03/15] arm64: kvm: Add build rules for separate nVHE object files
Hi David,
Quoted 108 lines · David Brazdil
Add new folder
arch/arm64/kvm/hyp/nvheand aMakefilefor building code that runs in EL2 under nVHE KVM.Compile each source file into a
.hyp.tmp.oobject first, then prefix all its symbols with "__kvm_nvhe_" usingobjcopyand produce a.hyp.o. Suffixes were chosen so that it would be possible for VHE and nVHE to share some source files, but compiled with different CFLAGS. nVHE build rules add -D__KVM_NVHE_HYPERVISOR__.The nVHE ELF symbol prefix is added to
kallsyms.cas ignored. EL2-only symbols will never appear in EL1 stack traces.Signed-off-by David Brazdil <[email protected]>--- arch/arm64/kernel/image-vars.h | 12 +++++++++++ arch/arm64/kvm/hyp/Makefile | 2 +- arch/arm64/kvm/hyp/nvhe/Makefile | 35 ++++++++++++++++++++++++++++++++ scripts/kallsyms.c | 1 + 4 files changed, 49 insertions(+), 1 deletion(-) create mode 100644 arch/arm64/kvm/hyp/nvhe/Makefilearch/arm64/kernel/image-vars.h +4
-0@@ -51,4 +51,16 @@ __efistub__ctype = _ctype;#endif +#ifdef CONFIG_KVM + +/* +* KVM nVHE code has its own symbol namespace prefixed by __kvm_nvhe_, to
+ * isolate it from the kernel proper. The following symbols are legally
+ * accessed by it, therefore provide aliases to make them linkable.
+ * Do not include symbols which may not be safely accessed under hypervisor
+ * memory mappings.
+ */arch/arm64/kernel/image-vars.h +3
-0+ +#endif /* CONFIG_KVM */ +#endif /* __ARM64_KERNEL_IMAGE_VARS_H */ arch/arm64/kvm/hyp/Makefile +1
-1@@ -6,7 +6,7 @@ ccflags-y += -fno-stack-protector -DDISABLE_BRANCH_PROFILING \ $(DISABLE_STACKLEAK_PLUGIN) -obj-$(CONFIG_KVM) += hyp.o +obj-$(CONFIG_KVM) += hyp.o nvhe/ obj-$(CONFIG_KVM_INDIRECT_VECTORS) += smccc_wa.o hyp-y := vgic-v3-sr.o timer-sr.o aarch32.o vgic-v2-cpuif-proxy.osysreg-sr.o \
arch/arm64/kvm/hyp/nvhe/Makefile new file +25
-0@@ -0,0 +1,35 @@ +# SPDX-License-Identifier: GPL-2.0 +# +# Makefile for Kernel-based Virtual Machine module, HYP/nVHE part +# + +asflags-y := -D__KVM_NVHE_HYPERVISOR__ +ccflags-y := -D__KVM_NVHE_HYPERVISOR__ -fno-stack-protector \ + -DDISABLE_BRANCH_PROFILING $(DISABLE_STACKLEAK_PLUGIN) + +obj-y := + +obj-y := $(patsubst %.o,%.hyp.o,$(obj-y)) +extra-y := $(patsubst %.hyp.o,%.hyp.tmp.o,$(obj-y)) + +$(obj)/%.hyp.tmp.o: $(src)/%.c FORCE + $(call if_changed_rule,cc_o_c) +$(obj)/%.hyp.tmp.o: $(src)/%.S FORCE + $(call if_changed_rule,as_o_S) +$(obj)/%.hyp.o: $(obj)/%.hyp.tmp.o FORCE + $(call if_changed,hypcopy) + +quiet_cmd_hypcopy = HYPCOPY $@ + cmd_hypcopy = $(OBJCOPY) --prefix-symbols=__kvm_nvhe_ $< $@ + +# KVM nVHE code is run at a different exception code with a differentmap, so
+# compiler instrumentation that inserts callbacks or checks into the code mayarch/arm64/kvm/hyp/nvhe/Makefile new file +7
-0+# cause crashes. Just disable it. +GCOV_PROFILE := n +KASAN_SANITIZE := n +UBSAN_SANITIZE := n +KCOV_INSTRUMENT := n + +# Skip objtool checking for this directory because nVHE code iscompiled with
arch/arm64/kvm/hyp/nvhe/Makefile new file +2
-0+# non-standard build rules. +OBJECT_FILES_NON_STANDARD := yscripts/kallsyms.c +0
-0
scripts/kallsyms.c static bool is_ignored_symbol(const char *name, · line 115
Quoted 7 lines
scripts/kallsyms.c +1
-0@@ -109,6 +109,7 @@ static bool is_ignored_symbol(const char *name, char type)".LASANPC" ,/* s390 kasan local symbols */ "__crc_" ,/* modversions */ "__efistub_" ,/* arm64 EFI stub namespace */ +"__kvm_nvhe_" ,/* arm64 non-VHE KVM namespace */ NULL };
I guess that one of the first use of this __KVM_NVHE_HYPERVISOR__ flag could be the has_vhe() predicate: if you're running the nVHE code, you are *guaranteed* not to use VHE at all.
Something like:
arch/arm64/include/asm/virt.h +2 -4
@@ -85,10 +85,8 @@ static inline bool is_kernel_in_hyp_mode(void) static __always_inline bool has_vhe(void) { -if (cpus_have_final_cap (ARM64_HAS_VIRT_HOST_EXTN )) -return true; - -return false; +return (__is_defined (__KVM_NVHE_HYPERVISOR__) && +cpus_have_final_cap (ARM64_HAS_VIRT_HOST_EXTN )); }#endif /* __ASSEMBLY__ */
Thanks,
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
Re: [PATCH v3 03/15] arm64: kvm: Add build rules for separate nVHE object files
Hey Marc,
Quoted 3 lines
I guess that one of the first use of this
__KVM_NVHE_HYPERVISOR__flag could be thehas_vhe()predicate: if you're running the nVHE code, you are *guaranteed* not to use VHE at all.
Sure, I'll add that to the next respin. If I'm reading __is_defined correctly, it means I have to define the flag as =1. Easy.
Thanks,
-David
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v3 4/15 arm64: kvm: Handle calls to prefixed hyp functions
In reply to David BrazdilFrom: Andrew Scull <[email protected]>
This patch is part of a series which builds KVM's non-VHE hyp code separately from VHE and the rest of the kernel.
Once hyp functions are moved to a hyp object, they will have prefixed symbols. This change declares and gets the address of the prefixed version for calls to the hyp functions.
To aid migration, the hyp functions that have not yet moved have their prefixed versions aliased to their non-prefixed version. This begins with all the hyp functions being listed and will reduce to none of them once the migration is complete.
Extracted kvm_call_hyp nVHE branches into own helper macros.
--- arch/arm64/include/asm/kvm_asm.h | 19 +++++++++++++++++++ arch/arm64/include/asm/kvm_host.h | 19 ++++++++++++++++--- arch/arm64/kernel/image-vars.h | 15 +++++++++++++++ 3 files changed, 50 insertions(+), 3 deletions(-)
arch/arm64/include/asm/kvm_asm.h +19 -0
@@ -42,6 +42,24 @@#include <linux/mm.h> +/* +* Translate name of a symbol defined in nVHE hyp to the name seen +* by kernel proper. All nVHE symbols are prefixed by the build system +* to avoid clashes with the VHE variants. +*/ +#define kvm_nvhe_sym (sym) __kvm_nvhe_##sym + +#define DECLARE_KVM_VHE_SYM (sym) extern char sym[] +#define DECLARE_KVM_NVHE_SYM (sym) extern char kvm_nvhe_sym(sym)[] + +/* +* Define a pair of symbols sharing the same name but one defined in +* VHE and the other in nVHE hyp implementations. +*/ +#define DECLARE_KVM_HYP_SYM (sym) \ + DECLARE_KVM_VHE_SYM(sym); \ + DECLARE_KVM_NVHE_SYM(sym) +/* Translate a kernel address of @sym into its equivalent linear mapping */ #define kvm_ksym_ref (sym) \ ({ \
@@ -50,6 +68,7 @@ val =lm_alias (&sym); \ val; \ }) +#define kvm_ksym_ref_nvhe (sym) kvm_ksym_ref(kvm_nvhe_sym(sym))struct kvm ;struct kvm_vcpu ;
arch/arm64/include/asm/kvm_host.h +16 -3
@@ -448,6 +448,20 @@ void kvm_arm_resume_guest(struct kvm *kvm);u64 __kvm_call_hyp (void *hypfn, ...); +#define kvm_call_hyp_nvhe (f, ...) \ + do { \ + DECLARE_KVM_NVHE_SYM(f); \ + __kvm_call_hyp(kvm_ksym_ref_nvhe(f), ##__VA_ARGS__); \ + } while(0) + +#define kvm_call_hyp_nvhe_ret (f, ...) \ + ({ \ + DECLARE_KVM_NVHE_SYM(f); \ + typeof(f(__VA_ARGS__)) ret; \ + ret = __kvm_call_hyp(kvm_ksym_ref_nvhe(f), \ + ##__VA_ARGS__); \ + }) +/* * The couple of isb() below are there to guarantee the same behaviour * on VHE as on !VHE, where the eret to EL1 acts as a context
@@ -459,7 +473,7 @@ u64 __kvm_call_hyp(void *hypfn, ...);f (__VA_ARGS__); \isb (); \ }else { \ - __kvm_call_hyp(kvm_ksym_ref(f), ##__VA_ARGS__); \ + kvm_call_hyp_nvhe(f, ##__VA_ARGS__); \ } \ }while (0 )
@@ -471,8 +485,7 @@ u64 __kvm_call_hyp(void *hypfn, ...); ret =f (__VA_ARGS__); \isb (); \ }else { \ - ret =__kvm_call_hyp (kvm_ksym_ref (f), \ - ##__VA_ARGS__); \ + ret = kvm_call_hyp_nvhe_ret(f, ##__VA_ARGS__); \ } \ \ ret; \
arch/arm64/kernel/image-vars.h +15 -0
@@ -61,6 +61,21 @@ __efistub__ctype = _ctype;* memory mappings. */ +__kvm_nvhe___kvm_enable_ssbs = __kvm_enable_ssbs; +__kvm_nvhe___kvm_flush_vm_context = __kvm_flush_vm_context; +__kvm_nvhe___kvm_get_mdcr_el2 = __kvm_get_mdcr_el2; +__kvm_nvhe___kvm_timer_set_cntvoff = __kvm_timer_set_cntvoff; +__kvm_nvhe___kvm_tlb_flush_local_vmid = __kvm_tlb_flush_local_vmid; +__kvm_nvhe___kvm_tlb_flush_vmid = __kvm_tlb_flush_vmid; +__kvm_nvhe___kvm_tlb_flush_vmid_ipa = __kvm_tlb_flush_vmid_ipa; +__kvm_nvhe___kvm_vcpu_run_nvhe = __kvm_vcpu_run_nvhe; +__kvm_nvhe___vgic_v3_get_ich_vtr_el2 = __vgic_v3_get_ich_vtr_el2; +__kvm_nvhe___vgic_v3_init_lrs = __vgic_v3_init_lrs; +__kvm_nvhe___vgic_v3_read_vmcr = __vgic_v3_read_vmcr; +__kvm_nvhe___vgic_v3_restore_aprs = __vgic_v3_restore_aprs; +__kvm_nvhe___vgic_v3_save_aprs = __vgic_v3_save_aprs; +__kvm_nvhe___vgic_v3_write_vmcr = __vgic_v3_write_vmcr; +#endif /* CONFIG_KVM */ #endif /* __ARM64_KERNEL_IMAGE_VARS_H */
Signature · 6 lines
2.27.0 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Marc Zyngier
maintainerRe: [PATCH v3 04/15] arm64: kvm: Handle calls to prefixed hyp functions
Hi David,
Quoted 14 lines · David Brazdil
From: Andrew Scull <[email protected]>
This patch is part of a series which builds KVM's non-VHE hyp code separately from VHE and the rest of the kernel.
Once hyp functions are moved to a hyp object, they will have prefixed symbols. This change declares and gets the address of the prefixed version for calls to the hyp functions.
To aid migration, the hyp functions that have not yet moved have their prefixed versions aliased to their non-prefixed version. This begins with all the hyp functions being listed and will reduce to none of them once the migration is complete.
Quoted 4 lines
Signed-off-by Andrew Scull <[email protected]>Extracted
kvm_call_hypnVHE branches into own helper macros.Signed-off-by David Brazdil <[email protected]>
nit: if you want to add this kind of comment, try to write it between square brackets, without blank lines in between:
Quoted 50 lines
--- arch/arm64/include/asm/kvm_asm.h | 19 +++++++++++++++++++ arch/arm64/include/asm/kvm_host.h | 19 ++++++++++++++++--- arch/arm64/kernel/image-vars.h | 15 +++++++++++++++ 3 files changed, 50 insertions(+), 3 deletions(-)arch/arm64/include/asm/kvm_asm.h +18
-0@@ -42,6 +42,24 @@#include <linux/mm.h> +/* +* Translate name of a symbol defined in nVHE hyp to the name seen +* by kernel proper. All nVHE symbols are prefixed by the build system +* to avoid clashes with the VHE variants. +*/ +#define kvm_nvhe_sym (sym) __kvm_nvhe_##sym + +#define DECLARE_KVM_VHE_SYM (sym) extern char sym[] +#define DECLARE_KVM_NVHE_SYM (sym) extern char kvm_nvhe_sym(sym)[] + +/* +* Define a pair of symbols sharing the same name but one defined in +* VHE and the other in nVHE hyp implementations. +*/ +#define DECLARE_KVM_HYP_SYM (sym) \ + DECLARE_KVM_VHE_SYM(sym); \ + DECLARE_KVM_NVHE_SYM(sym) +/* Translate a kernel address of @sym into its equivalent linear mapping */
#definekvm_ksym_ref(sym)\arch/arm64/include/asm/kvm_asm.h +1
-0({ \@@ -50,6 +68,7 @@ val =lm_alias (&sym); \ val; \ }) +#define kvm_ksym_ref_nvhe (sym) kvm_ksym_ref(kvm_nvhe_sym(sym))struct kvm ;struct kvm_vcpu ;arch/arm64/include/asm/kvm_host.h +0
-0
arch/arm64/include/asm/kvm_host.h void kvm_arm_resume_guest(struct kvm *kvm); · line 453
Quoted 6 lines
arch/arm64/include/asm/kvm_host.h +3
-0@@ -448,6 +448,20 @@ void kvm_arm_resume_guest(struct kvm *kvm);u64 __kvm_call_hyp (void *hypfn, ...); +#define kvm_call_hyp_nvhe (f, ...) \ + do { \ + DECLARE_KVM_NVHE_SYM(f); \
I wanted to move this out to __kvm_call_hyp, but the nVHE ssbs code got in the way... Oh well.
arch/arm64/include/asm/kvm_host.h void kvm_arm_resume_guest(struct kvm *kvm); · line 462
Quoted 9 lines
arch/arm64/include/asm/kvm_host.h +9
-0+ __kvm_call_hyp(kvm_ksym_ref_nvhe(f), ##__VA_ARGS__); \ + } while(0) + +#define kvm_call_hyp_nvhe_ret (f, ...) \ + ({ \ + DECLARE_KVM_NVHE_SYM(f); \ + typeof(f(__VA_ARGS__)) ret; \ + ret = __kvm_call_hyp(kvm_ksym_ref_nvhe(f), \ + ##__VA_ARGS__); \
You don't need to redefine ret here. actually, you can just evaluate the expression and let C do its magic:
arch/arm64/include/asm/kvm_host.h +1 -3
@@ -457,9 +457,7 @@ u64 __kvm_call_hyp(void *hypfn, ...);#define kvm_call_hyp_nvhe_ret (f, ...) \ ({ \ DECLARE_KVM_NVHE_SYM(f); \ - typeof(f(__VA_ARGS__)) ret; \ - ret = __kvm_call_hyp(kvm_ksym_ref_nvhe(f), \ - ##__VA_ARGS__); \ + __kvm_call_hyp(kvm_ksym_ref_nvhe(f), ##__VA_ARGS__); \ })/*
Quoted 40 lines
arch/arm64/include/asm/kvm_host.h +4
-3+ }) +/* * The couple of isb() below are there to guarantee the same behaviour * on VHE as on !VHE, where the eret to EL1 acts as a context @@ -459,7 +473,7 @@ u64 __kvm_call_hyp(void *hypfn, ...);f (__VA_ARGS__); \isb (); \ }else { \ - __kvm_call_hyp(kvm_ksym_ref(f), ##__VA_ARGS__); \ + kvm_call_hyp_nvhe(f, ##__VA_ARGS__); \ } \ }while (0 )@@ -471,8 +485,7 @@ u64 __kvm_call_hyp(void *hypfn, ...); ret =f (__VA_ARGS__); \isb (); \ }else { \ - ret =__kvm_call_hyp (kvm_ksym_ref (f), \ - ##__VA_ARGS__); \ + ret = kvm_call_hyp_nvhe_ret(f, ##__VA_ARGS__); \ } \ \ ret; \arch/arm64/kernel/image-vars.h +8
-0@@ -61,6 +61,21 @@ __efistub__ctype = _ctype;* memory mappings. */ +__kvm_nvhe___kvm_enable_ssbs = __kvm_enable_ssbs; +__kvm_nvhe___kvm_flush_vm_context = __kvm_flush_vm_context; +__kvm_nvhe___kvm_get_mdcr_el2 = __kvm_get_mdcr_el2; +__kvm_nvhe___kvm_timer_set_cntvoff = __kvm_timer_set_cntvoff; +__kvm_nvhe___kvm_tlb_flush_local_vmid = __kvm_tlb_flush_local_vmid; +__kvm_nvhe___kvm_tlb_flush_vmid = __kvm_tlb_flush_vmid; +__kvm_nvhe___kvm_tlb_flush_vmid_ipa = __kvm_tlb_flush_vmid_ipa; +__kvm_nvhe___kvm_vcpu_run_nvhe = __kvm_vcpu_run_nvhe;
arch/arm64/kernel/image-vars.h __efistub__ctype = _ctype; · line 81
Quoted 10 lines
arch/arm64/kernel/image-vars.h +7
-0+__kvm_nvhe___vgic_v3_get_ich_vtr_el2 = __vgic_v3_get_ich_vtr_el2; +__kvm_nvhe___vgic_v3_init_lrs = __vgic_v3_init_lrs; +__kvm_nvhe___vgic_v3_read_vmcr = __vgic_v3_read_vmcr; +__kvm_nvhe___vgic_v3_restore_aprs = __vgic_v3_restore_aprs; +__kvm_nvhe___vgic_v3_save_aprs = __vgic_v3_save_aprs; +__kvm_nvhe___vgic_v3_write_vmcr = __vgic_v3_write_vmcr; +#endif /* CONFIG_KVM */ #endif /* __ARM64_KERNEL_IMAGE_VARS_H */
Otherwise looks good.
Thanks,
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
kernel test robot
build botRe: [PATCH v3 04/15] arm64: kvm: Handle calls to prefixed hyp functions
In reply to David BrazdilThank you for the patch! Perhaps something to improve:
Hi David,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on v5.8-rc1 next-20200618]
[cannot apply to kvmarm/next arm64/for-next/core arm-perf/for-next/perf] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/David-Brazdil/Split-off-nVHE-hyp-code/20200618-203230 base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 1b5044021070efa3259f3e9548dc35d1eb6aa844 config: arm64-allyesconfig (attached as .config) compiler: aarch64-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=arm64
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>, old ones prefixed by <<):
In file included from include/linux/kvm_host.h:36, from arch/arm64/kvm/arm.c:11:arch/arm64/kvm/arm.c: In function 'kvm_arch_vcpu_ioctl_run':
>> arch/arm64/include/asm/kvm_host.h:460:26: warning: variable 'ret' set but not used [-Wunused-but-set-variable] 460 | typeof(f(__VA_ARGS__)) ret; | ^~~ >> arch/arm64/include/asm/kvm_host.h:488:10: note: in expansion of macro 'kvm_call_hyp_nvhe_ret' 488 | ret = kvm_call_hyp_nvhe_ret(f, ##__VA_ARGS__); | ^~~~~~~~~~~~~~~~~~~~~ >> arch/arm64/kvm/arm.c:754:10: note: in expansion of macro 'kvm_call_hyp_ret'
754 | ret = kvm_call_hyp_ret(__kvm_vcpu_run_nvhe, vcpu);
| ^~~~~~~~~~~~~~~~
--
In file included from arch/arm64/include/asm/percpu.h:228, from arch/arm64/include/asm/smp.h:28,
from include/linux/smp.h:89,
from include/linux/percpu.h:7,
from include/linux/context_tracking_state.h:5, from include/linux/hardirq.h:5,
from include/linux/kvm_host.h:7,
from arch/arm64/kvm/debug.c:9:arch/arm64/kvm/debug.c: In function 'kvm_arm_init_debug':
>> arch/arm64/include/asm/kvm_host.h:460:26: warning: variable 'ret' set but not used [-Wunused-but-set-variable]
460 | typeof(f(__VA_ARGS__)) ret; | ^~~include/asm-generic/percpu.h:72:26: note: in definition of macro 'raw_cpu_generic_to_op'
72 | *raw_cpu_ptr(&(pcp)) op val; | ^~~include/linux/percpu-defs.h:377:11: note: in expansion of macro 'raw_cpu_write_1'
377 | case 1: stem##1(variable, __VA_ARGS__);break; | ^~~~include/linux/percpu-defs.h:421:34: note: in expansion of macro '__pcpu_size_call' 421 | #define raw_cpu_write(pcp, val) __pcpu_size_call(raw_cpu_write_, pcp, val)
| ^~~~~~~~~~~~~~~~include/linux/percpu-defs.h:452:2: note: in expansion of macro 'raw_cpu_write'
452 | raw_cpu_write(pcp, val); | ^~~~~~~~~~~~~
>> arch/arm64/kvm/debug.c:68:2: note: in expansion of macro '__this_cpu_write' 68 | __this_cpu_write(mdcr_el2, kvm_call_hyp_ret(__kvm_get_mdcr_el2)); | ^~~~~~~~~~~~~~~~ >> arch/arm64/include/asm/kvm_host.h:488:10: note: in expansion of macro 'kvm_call_hyp_nvhe_ret' 488 | ret = kvm_call_hyp_nvhe_ret(f, ##__VA_ARGS__); | ^~~~~~~~~~~~~~~~~~~~~ >> arch/arm64/kvm/debug.c:68:29: note: in expansion of macro 'kvm_call_hyp_ret' 68 | __this_cpu_write(mdcr_el2, kvm_call_hyp_ret(__kvm_get_mdcr_el2)); | ^~~~~~~~~~~~~~~~ >> arch/arm64/include/asm/kvm_host.h:460:26: warning: variable 'ret' set but not used [-Wunused-but-set-variable]
460 | typeof(f(__VA_ARGS__)) ret; | ^~~include/asm-generic/percpu.h:72:26: note: in definition of macro 'raw_cpu_generic_to_op'
72 | *raw_cpu_ptr(&(pcp)) op val; | ^~~include/linux/percpu-defs.h:378:11: note: in expansion of macro 'raw_cpu_write_2'
378 | case 2: stem##2(variable, __VA_ARGS__);break; | ^~~~include/linux/percpu-defs.h:421:34: note: in expansion of macro '__pcpu_size_call' 421 | #define raw_cpu_write(pcp, val) __pcpu_size_call(raw_cpu_write_, pcp, val)
| ^~~~~~~~~~~~~~~~include/linux/percpu-defs.h:452:2: note: in expansion of macro 'raw_cpu_write'
452 | raw_cpu_write(pcp, val); | ^~~~~~~~~~~~~
>> arch/arm64/kvm/debug.c:68:2: note: in expansion of macro '__this_cpu_write' 68 | __this_cpu_write(mdcr_el2, kvm_call_hyp_ret(__kvm_get_mdcr_el2)); | ^~~~~~~~~~~~~~~~ >> arch/arm64/include/asm/kvm_host.h:488:10: note: in expansion of macro 'kvm_call_hyp_nvhe_ret' 488 | ret = kvm_call_hyp_nvhe_ret(f, ##__VA_ARGS__); | ^~~~~~~~~~~~~~~~~~~~~ >> arch/arm64/kvm/debug.c:68:29: note: in expansion of macro 'kvm_call_hyp_ret' 68 | __this_cpu_write(mdcr_el2, kvm_call_hyp_ret(__kvm_get_mdcr_el2)); | ^~~~~~~~~~~~~~~~ >> arch/arm64/include/asm/kvm_host.h:460:26: warning: variable 'ret' set but not used [-Wunused-but-set-variable]
460 | typeof(f(__VA_ARGS__)) ret; | ^~~include/asm-generic/percpu.h:72:26: note: in definition of macro 'raw_cpu_generic_to_op'
72 | *raw_cpu_ptr(&(pcp)) op val; | ^~~include/linux/percpu-defs.h:379:11: note: in expansion of macro 'raw_cpu_write_4'
379 | case 4: stem##4(variable, __VA_ARGS__);break; | ^~~~include/linux/percpu-defs.h:421:34: note: in expansion of macro '__pcpu_size_call' 421 | #define raw_cpu_write(pcp, val) __pcpu_size_call(raw_cpu_write_, pcp, val)
| ^~~~~~~~~~~~~~~~include/linux/percpu-defs.h:452:2: note: in expansion of macro 'raw_cpu_write'
452 | raw_cpu_write(pcp, val); | ^~~~~~~~~~~~~
>> arch/arm64/kvm/debug.c:68:2: note: in expansion of macro '__this_cpu_write' 68 | __this_cpu_write(mdcr_el2, kvm_call_hyp_ret(__kvm_get_mdcr_el2)); | ^~~~~~~~~~~~~~~~ >> arch/arm64/include/asm/kvm_host.h:488:10: note: in expansion of macro 'kvm_call_hyp_nvhe_ret' 488 | ret = kvm_call_hyp_nvhe_ret(f, ##__VA_ARGS__); | ^~~~~~~~~~~~~~~~~~~~~ >> arch/arm64/kvm/debug.c:68:29: note: in expansion of macro 'kvm_call_hyp_ret' 68 | __this_cpu_write(mdcr_el2, kvm_call_hyp_ret(__kvm_get_mdcr_el2)); | ^~~~~~~~~~~~~~~~ >> arch/arm64/include/asm/kvm_host.h:460:26: warning: variable 'ret' set but not used [-Wunused-but-set-variable]
460 | typeof(f(__VA_ARGS__)) ret; | ^~~include/asm-generic/percpu.h:72:26: note: in definition of macro 'raw_cpu_generic_to_op'
72 | *raw_cpu_ptr(&(pcp)) op val; | ^~~include/linux/percpu-defs.h:380:11: note: in expansion of macro 'raw_cpu_write_8'
380 | case 8: stem##8(variable, __VA_ARGS__);break; | ^~~~include/linux/percpu-defs.h:421:34: note: in expansion of macro '__pcpu_size_call' 421 | #define raw_cpu_write(pcp, val) __pcpu_size_call(raw_cpu_write_, pcp, val)
| ^~~~~~~~~~~~~~~~include/linux/percpu-defs.h:452:2: note: in expansion of macro 'raw_cpu_write'
452 | raw_cpu_write(pcp, val); | ^~~~~~~~~~~~~
>> arch/arm64/kvm/debug.c:68:2: note: in expansion of macro '__this_cpu_write' 68 | __this_cpu_write(mdcr_el2, kvm_call_hyp_ret(__kvm_get_mdcr_el2)); | ^~~~~~~~~~~~~~~~ >> arch/arm64/include/asm/kvm_host.h:488:10: note: in expansion of macro 'kvm_call_hyp_nvhe_ret' 488 | ret = kvm_call_hyp_nvhe_ret(f, ##__VA_ARGS__); | ^~~~~~~~~~~~~~~~~~~~~ >> arch/arm64/kvm/debug.c:68:29: note: in expansion of macro 'kvm_call_hyp_ret'
68 | __this_cpu_write(mdcr_el2, kvm_call_hyp_ret(__kvm_get_mdcr_el2));
| ^~~~~~~~~~~~~~~~
--
In file included from include/linux/kvm_host.h:36,
from arch/arm64/kvm/vgic/vgic-v3.c:5:arch/arm64/kvm/vgic/vgic-v3.c: In function 'vgic_v3_probe':
>> arch/arm64/include/asm/kvm_host.h:460:26: warning: variable 'ret' set but not used [-Wunused-but-set-variable] 460 | typeof(f(__VA_ARGS__)) ret; | ^~~ >> arch/arm64/include/asm/kvm_host.h:488:10: note: in expansion of macro 'kvm_call_hyp_nvhe_ret' 488 | ret = kvm_call_hyp_nvhe_ret(f, ##__VA_ARGS__); | ^~~~~~~~~~~~~~~~~~~~~ >> arch/arm64/kvm/vgic/vgic-v3.c:586:20: note: in expansion of macro 'kvm_call_hyp_ret' 586 | u32 ich_vtr_el2 = kvm_call_hyp_ret(__vgic_v3_get_ich_vtr_el2); | ^~~~~~~~~~~~~~~~ arch/arm64/kvm/vgic/vgic-v3.c: In function 'vgic_v3_vmcr_sync': >> arch/arm64/include/asm/kvm_host.h:460:26: warning: variable 'ret' set but not used [-Wunused-but-set-variable] 460 | typeof(f(__VA_ARGS__)) ret; | ^~~ >> arch/arm64/include/asm/kvm_host.h:488:10: note: in expansion of macro 'kvm_call_hyp_nvhe_ret'
488 | ret = kvm_call_hyp_nvhe_ret(f, ##__VA_ARGS__); | ^~~~~~~~~~~~~~~~~~~~~arch/arm64/kvm/vgic/vgic-v3.c:678:23: note: in expansion of macro 'kvm_call_hyp_ret'
678 | cpu_if->vgic_vmcr = kvm_call_hyp_ret(__vgic_v3_read_vmcr);
| ^~~~~~~~~~~~~~~~
vim +/ret +460 arch/arm64/include/asm/kvm_host.h
450
451 #define kvm_call_hyp_nvhe(f, ...) \
452 do { \
453 DECLARE_KVM_NVHE_SYM(f); \
454 __kvm_call_hyp(kvm_ksym_ref_nvhe(f), ##__VA_ARGS__); \
455 } while(0)
456
457 #define kvm_call_hyp_nvhe_ret(f, ...) \
458 ({ \
459 DECLARE_KVM_NVHE_SYM(f); \
> 460 typeof(f(__VA_ARGS__)) ret; \
461 ret = __kvm_call_hyp(kvm_ksym_ref_nvhe(f), \
462 ##__VA_ARGS__); \
463 })
464
465 /*
466 * The couple of isb() below are there to guarantee the same behaviour
467 * on VHE as on !VHE, where the eret to EL1 acts as a context
468 * synchronization event.
469 */
470 #define kvm_call_hyp(f, ...) \
471 do { \
472 if (has_vhe()) { \
473 f(__VA_ARGS__); \
474 isb(); \
475 } else { \
476 kvm_call_hyp_nvhe(f, ##__VA_ARGS__); \
477 } \
478 } while(0)
479
480 #define kvm_call_hyp_ret(f, ...) \
481 ({ \
482 typeof(f(__VA_ARGS__)) ret; \
483 \
484 if (has_vhe()) { \
485 ret = f(__VA_ARGS__); \
486 isb(); \
487 } else { \
> 488 ret = kvm_call_hyp_nvhe_ret(f, ##__VA_ARGS__); \ 489 } \
490 \
491 ret; \
492 })
493
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
PATCH v3 5/15 arm64: kvm: Build hyp-entry.S separately for VHE/nVHE
In reply to David BrazdilThis patch is part of a series which builds KVM's non-VHE hyp code separately from VHE and the rest of the kernel.
hyp-entry.S contains implementation of KVM hyp vectors. This code is mostly shared between VHE/nVHE, therefore compile it under both VHE and nVHE build rules. nVHE-specific host HVC handler is hidden behind __KVM_NVHE_HYPERVISOR__.
Adjust code which selects which KVM hyp vecs to install to choose the correct VHE/nVHE symbol.
--- arch/arm64/include/asm/kvm_asm.h | 7 ++++++- arch/arm64/include/asm/kvm_mmu.h | 16 ++++++++++------ arch/arm64/include/asm/mmu.h | 7 ------- arch/arm64/kernel/cpu_errata.c | 4 +++- arch/arm64/kernel/image-vars.h | 12 ++++++++++++ arch/arm64/kvm/hyp/hyp-entry.S | 2 ++ arch/arm64/kvm/hyp/nvhe/Makefile | 2 +- arch/arm64/kvm/va_layout.c | 2 +- 8 files changed, 35 insertions(+), 17 deletions(-)
arch/arm64/include/asm/kvm_asm.h +6 -1
@@ -76,7 +76,12 @@ struct kvm_vcpu;extern char __kvm_hyp_init[];extern char __kvm_hyp_init_end[]; -extern char __kvm_hyp_vector[]; +DECLARE_KVM_HYP_SYM (__kvm_hyp_vector); + +#ifdef CONFIG_KVM_INDIRECT_VECTORS +DECLARE_KVM_HYP_SYM (__bp_harden_hyp_vecs); +extern atomic_t arm64_el2_vector_last_slot; +#endif extern void __kvm_flush_vm_context (void );extern void __kvm_tlb_flush_vmid_ipa (struct kvm *kvm,phys_addr_t ipa);
arch/arm64/include/asm/kvm_mmu.h +10 -6
@@ -461,11 +461,15 @@ extern int __kvm_harden_el2_vector_slot;static inline void *kvm_get_hyp_vector (void ) {struct bp_hardening_data *data =arm64_get_bp_hardening_data (); -void *vect =kern_hyp_va (kvm_ksym_ref (__kvm_hyp_vector));int slot =-1 ; +void *vect =kern_hyp_va (has_vhe () + ?kvm_ksym_ref (__kvm_hyp_vector) + :kvm_ksym_ref_nvhe (__kvm_hyp_vector));if (cpus_have_const_cap (ARM64_HARDEN_BRANCH_PREDICTOR ) && data->fn) { - vect =kern_hyp_va (kvm_ksym_ref (__bp_harden_hyp_vecs)); + vect =kern_hyp_va (has_vhe () + ?kvm_ksym_ref (__bp_harden_hyp_vecs) + :kvm_ksym_ref_nvhe (__bp_harden_hyp_vecs)); slot = data->hyp_vectors_slot; }
@@ -494,12 +498,11 @@ static inline int kvm_map_vectors(void)* HBP + HEL2 -> use hardened vertors and use exec mapping */ if (cpus_have_const_cap (ARM64_HARDEN_BRANCH_PREDICTOR )) { - __kvm_bp_vect_base =kvm_ksym_ref (__bp_harden_hyp_vecs); - __kvm_bp_vect_base =kern_hyp_va (__kvm_bp_vect_base); + __kvm_bp_vect_base =kern_hyp_va (kvm_ksym_ref_nvhe (__bp_harden_hyp_vecs)); }if (cpus_have_const_cap (ARM64_HARDEN_EL2_VECTORS )) { -phys_addr_t vect_pa =__pa_symbol (__bp_harden_hyp_vecs); +phys_addr_t vect_pa =__pa_symbol (kvm_nvhe_sym (__bp_harden_hyp_vecs));unsigned long size = __BP_HARDEN_HYP_VECS_SZ;/*
@@ -518,7 +521,8 @@ static inline int kvm_map_vectors(void)#else static inline void *kvm_get_hyp_vector (void ) { -return kern_hyp_va (kvm_ksym_ref (__kvm_hyp_vector)); +return kern_hyp_va (has_vhe () ?kvm_ksym_ref (__kvm_hyp_vector) + :kvm_ksym_ref_nvhe (__kvm_hyp_vector)); }static inline int kvm_map_vectors (void )
arch/arm64/include/asm/mmu.h +0 -7
@@ -42,13 +42,6 @@ struct bp_hardening_data {bp_hardening_cb_t fn; }; -#if (defined(CONFIG_HARDEN_BRANCH_PREDICTOR ) || \ - defined(CONFIG_HARDEN_EL2_VECTORS )) - -extern char __bp_harden_hyp_vecs[]; -extern atomic_t arm64_el2_vector_last_slot; -#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR || CONFIG_HARDEN_EL2_VECTORS */ -#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR DECLARE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data);
arch/arm64/kernel/cpu_errata.c +3 -1
@@ -117,7 +117,9 @@ DEFINE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data);static void __copy_hyp_vect_bpi (int slot,const char *hyp_vecs_start,const char *hyp_vecs_end) { -void *dst =lm_alias (__bp_harden_hyp_vecs + slot *SZ_2K ); +char *vec =has_vhe () ? __bp_harden_hyp_vecs + :kvm_nvhe_sym (__bp_harden_hyp_vecs); +void *dst =lm_alias (vec + slot *SZ_2K );int i;for (i =0 ; i <SZ_2K ; i +=0x80 )
arch/arm64/kernel/image-vars.h +12 -0
@@ -61,9 +61,11 @@ __efistub__ctype = _ctype;* memory mappings. */ +__kvm_nvhe___guest_exit = __guest_exit; __kvm_nvhe___kvm_enable_ssbs = __kvm_enable_ssbs; __kvm_nvhe___kvm_flush_vm_context = __kvm_flush_vm_context; __kvm_nvhe___kvm_get_mdcr_el2 = __kvm_get_mdcr_el2; +__kvm_nvhe___kvm_handle_stub_hvc = __kvm_handle_stub_hvc; __kvm_nvhe___kvm_timer_set_cntvoff = __kvm_timer_set_cntvoff; __kvm_nvhe___kvm_tlb_flush_local_vmid = __kvm_tlb_flush_local_vmid; __kvm_nvhe___kvm_tlb_flush_vmid = __kvm_tlb_flush_vmid;
@@ -75,6 +77,16 @@ __kvm_nvhe___vgic_v3_read_vmcr = __vgic_v3_read_vmcr; __kvm_nvhe___vgic_v3_restore_aprs = __vgic_v3_restore_aprs; __kvm_nvhe___vgic_v3_save_aprs = __vgic_v3_save_aprs; __kvm_nvhe___vgic_v3_write_vmcr = __vgic_v3_write_vmcr; +__kvm_nvhe_abort_guest_exit_end = abort_guest_exit_end; +__kvm_nvhe_abort_guest_exit_start = abort_guest_exit_start; +__kvm_nvhe_arm64_enable_wa2_handling = arm64_enable_wa2_handling; +__kvm_nvhe_arm64_ssbd_callback_required = arm64_ssbd_callback_required; +__kvm_nvhe_hyp_panic = hyp_panic; +__kvm_nvhe_kimage_voffset = kimage_voffset; +__kvm_nvhe_kvm_host_data = kvm_host_data; +__kvm_nvhe_kvm_patch_vector_branch = kvm_patch_vector_branch; +__kvm_nvhe_kvm_update_va_mask = kvm_update_va_mask; +__kvm_nvhe_panic = panic;#endif /* CONFIG_KVM */
arch/arm64/kvm/hyp/hyp-entry.S +2 -0
@@ -40,6 +40,7 @@ el1_sync: // Guest trapped into EL2 ccmp x0, #ESR_ELx_EC_HVC32, #4, ne b.ne el1_trap +#ifdef __KVM_NVHE_HYPERVISOR__ mrs x1, vttbr_el2 // If vttbr is valid, the guest cbnz x1, el1_hvc_guest // called HVC
@@ -74,6 +75,7 @@ el1_sync: // Guest trapped into EL2 eret sb +#endif /* __KVM_NVHE_HYPERVISOR__ */ el1_hvc_guest: /*
arch/arm64/kvm/hyp/nvhe/Makefile +1 -1
@@ -7,7 +7,7 @@ asflags-y := -D__KVM_NVHE_HYPERVISOR__ ccflags-y := -D__KVM_NVHE_HYPERVISOR__ -fno-stack-protector \ -DDISABLE_BRANCH_PROFILING $(DISABLE_STACKLEAK_PLUGIN) -obj-y := +obj-y := ../hyp-entry.o obj-y := $(patsubst %.o,%.hyp.o,$(obj-y)) extra-y := $(patsubst %.hyp.o,%.hyp.tmp.o,$(obj-y))
arch/arm64/kvm/va_layout.c +1 -1
@@ -150,7 +150,7 @@ void kvm_patch_vector_branch(struct alt_instr *alt,/* * Compute HYP VA by using the same computation as kern_hyp_va() */ - addr = (uintptr_t )kvm_ksym_ref (__kvm_hyp_vector); + addr = (uintptr_t )kvm_ksym_ref_nvhe (__kvm_hyp_vector); addr &= va_mask; addr |= tag_val << tag_lsb;
Signature · 6 lines
2.27.0 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Marc Zyngier
maintainerRe: [PATCH v3 05/15] arm64: kvm: Build hyp-entry.S separately for VHE/nVHE
Hi David,
Quoted 2 lines · David Brazdil
This patch is part of a series which builds KVM's non-VHE hyp code separately from VHE and the rest of the kernel.
The above comment doesn't really belong here, and us only fit for the cover letter.
Quoted 44 lines
hyp-entry.Scontains implementation of KVM hyp vectors. This code is mostly shared between VHE/nVHE, therefore compile it under both VHE and nVHE build rules. nVHE-specific host HVC handler is hidden behind__KVM_NVHE_HYPERVISOR__.Adjust code which selects which KVM hyp vecs to install to choose the correct VHE/nVHE symbol.
Signed-off-by David Brazdil <[email protected]>--- arch/arm64/include/asm/kvm_asm.h | 7 ++++++- arch/arm64/include/asm/kvm_mmu.h | 16 ++++++++++------ arch/arm64/include/asm/mmu.h | 7 ------- arch/arm64/kernel/cpu_errata.c | 4 +++- arch/arm64/kernel/image-vars.h | 12 ++++++++++++ arch/arm64/kvm/hyp/hyp-entry.S | 2 ++ arch/arm64/kvm/hyp/nvhe/Makefile | 2 +- arch/arm64/kvm/va_layout.c | 2 +- 8 files changed, 35 insertions(+), 17 deletions(-)arch/arm64/include/asm/kvm_asm.h +6
-1@@ -76,7 +76,12 @@ struct kvm_vcpu;extern char __kvm_hyp_init[];extern char __kvm_hyp_init_end[]; -extern char __kvm_hyp_vector[]; +DECLARE_KVM_HYP_SYM (__kvm_hyp_vector); + +#ifdef CONFIG_KVM_INDIRECT_VECTORS +DECLARE_KVM_HYP_SYM (__bp_harden_hyp_vecs); +extern atomic_t arm64_el2_vector_last_slot; +#endif extern void __kvm_flush_vm_context (void ); extern void __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_tipa);
arch/arm64/include/asm/kvm_mmu.h +0
-0
arch/arm64/include/asm/kvm_mmu.h extern int __kvm_harden_el2_vector_slot; · line 467
Quoted 8 lines
arch/arm64/include/asm/kvm_mmu.h +3
-1@@ -461,11 +461,15 @@ extern int __kvm_harden_el2_vector_slot;static inline void *kvm_get_hyp_vector (void ) {struct bp_hardening_data *data =arm64_get_bp_hardening_data (); -void *vect =kern_hyp_va (kvm_ksym_ref (__kvm_hyp_vector));int slot =-1 ; +void *vect =kern_hyp_va (has_vhe () + ?kvm_ksym_ref (__kvm_hyp_vector) + :kvm_ksym_ref_nvhe (__kvm_hyp_vector));
If you are introducing has_vhe() at this stage, then you might as well not apply kernel_hyp_va() to the address. This also make the declaration block look a bit ugly (yes, I'm a bit of a maniac). I'd rather see something like:
arch/arm64/include/asm/kvm_mmu.h +12 -6
@@ -457,19 +457,25 @@ static inline int kvm_write_guest_lock(struct kvm *kvm, gpa_t gpa,extern void *__kvm_bp_vect_base;extern int __kvm_harden_el2_vector_slot; +#define get_hyp_vector_address (v) \ +({ \ + void *__v; \ + if (has_vhe()) \ + __v = kvm_ksym_ref(v); \ + else \ + __v = kern_hyp_va(kvm_ksym_ref_nvhe(v)); \ + __v; \ +}) +/* This is called on both VHE and !VHE systems */ static inline void *kvm_get_hyp_vector (void ) {struct bp_hardening_data *data =arm64_get_bp_hardening_data (); +void *vect =get_hyp_vector_address (__kvm_hyp_vector);int slot =-1 ; -void *vect =kern_hyp_va (has_vhe () - ?kvm_ksym_ref (__kvm_hyp_vector) - :kvm_ksym_ref_nvhe (__kvm_hyp_vector));if (cpus_have_const_cap (ARM64_HARDEN_BRANCH_PREDICTOR ) && data->fn) { - vect =kern_hyp_va (has_vhe () - ?kvm_ksym_ref (__bp_harden_hyp_vecs) - :kvm_ksym_ref_nvhe (__bp_harden_hyp_vecs)); + vect =get_hyp_vector_address (__bp_harden_hyp_vecs); slot = data->hyp_vectors_slot; }
Quoted 61 lines
arch/arm64/include/asm/kvm_mmu.h +5
-4if (cpus_have_const_cap (ARM64_HARDEN_BRANCH_PREDICTOR ) && data->fn) { - vect =kern_hyp_va (kvm_ksym_ref (__bp_harden_hyp_vecs)); + vect =kern_hyp_va (has_vhe () + ?kvm_ksym_ref (__bp_harden_hyp_vecs) + :kvm_ksym_ref_nvhe (__bp_harden_hyp_vecs)); slot = data->hyp_vectors_slot; }@@ -494,12 +498,11 @@ static inline int kvm_map_vectors(void)* HBP + HEL2 -> use hardened vertors and use exec mapping */ if (cpus_have_const_cap (ARM64_HARDEN_BRANCH_PREDICTOR )) { - __kvm_bp_vect_base =kvm_ksym_ref (__bp_harden_hyp_vecs); - __kvm_bp_vect_base =kern_hyp_va (__kvm_bp_vect_base); + __kvm_bp_vect_base =kern_hyp_va (kvm_ksym_ref_nvhe (__bp_harden_hyp_vecs)); }if (cpus_have_const_cap (ARM64_HARDEN_EL2_VECTORS )) { -phys_addr_t vect_pa =__pa_symbol (__bp_harden_hyp_vecs); +phys_addr_t vect_pa =__pa_symbol (kvm_nvhe_sym (__bp_harden_hyp_vecs));unsigned long size = __BP_HARDEN_HYP_VECS_SZ;arch/arm64/include/asm/kvm_mmu.h +2
-1/* @@ -518,7 +521,8 @@ static inline int kvm_map_vectors(void)#else static inline void *kvm_get_hyp_vector (void ) { -return kern_hyp_va (kvm_ksym_ref (__kvm_hyp_vector)); +return kern_hyp_va (has_vhe () ?kvm_ksym_ref (__kvm_hyp_vector) + :kvm_ksym_ref_nvhe (__kvm_hyp_vector)); }static inline int kvm_map_vectors (void )arch/arm64/include/asm/mmu.h +0
-7@@ -42,13 +42,6 @@ struct bp_hardening_data {bp_hardening_cb_t fn; }; -#if (defined(CONFIG_HARDEN_BRANCH_PREDICTOR ) || \ - defined(CONFIG_HARDEN_EL2_VECTORS )) - -extern char __bp_harden_hyp_vecs[]; -extern atomic_t arm64_el2_vector_last_slot; -#endif /* CONFIG_HARDEN_BRANCH_PREDICTOR || CONFIG_HARDEN_EL2_VECTORS */ -#ifdef CONFIG_HARDEN_BRANCH_PREDICTOR DECLARE_PER_CPU_READ_MOSTLY(struct bp_hardening_data,
bp_hardening_data);arch/arm64/kernel/cpu_errata.c +0
-0
arch/arm64/kernel/cpu_errata.c DEFINE_PER_CPU_READ_MOSTLY(struct · line 122
Quoted 7 lines
arch/arm64/kernel/cpu_errata.c +2
-1@@ -117,7 +117,9 @@ DEFINE_PER_CPU_READ_MOSTLY(struct bp_hardening_data, bp_hardening_data);static void __copy_hyp_vect_bpi (int slot,const char *hyp_vecs_start,const char *hyp_vecs_end) { -void *dst =lm_alias (__bp_harden_hyp_vecs + slot *SZ_2K ); +char *vec =has_vhe () ? __bp_harden_hyp_vecs + :kvm_nvhe_sym (__bp_harden_hyp_vecs);
If we get this construct often, then something that abstracts the uggliness of the symbol duality would be nice...
Quoted 77 lines
arch/arm64/kernel/cpu_errata.c +1
-0+void *dst =lm_alias (vec + slot *SZ_2K );int i;arch/arm64/kernel/cpu_errata.c +0
-0for (i =0 ; i <SZ_2K ; i +=0x80 )arch/arm64/kernel/image-vars.h +12
-0@@ -61,9 +61,11 @@ __efistub__ctype = _ctype;* memory mappings. */ +__kvm_nvhe___guest_exit = __guest_exit; __kvm_nvhe___kvm_enable_ssbs = __kvm_enable_ssbs; __kvm_nvhe___kvm_flush_vm_context = __kvm_flush_vm_context; __kvm_nvhe___kvm_get_mdcr_el2 = __kvm_get_mdcr_el2; +__kvm_nvhe___kvm_handle_stub_hvc = __kvm_handle_stub_hvc; __kvm_nvhe___kvm_timer_set_cntvoff = __kvm_timer_set_cntvoff; __kvm_nvhe___kvm_tlb_flush_local_vmid = __kvm_tlb_flush_local_vmid; __kvm_nvhe___kvm_tlb_flush_vmid = __kvm_tlb_flush_vmid;@@ -75,6 +77,16 @@ __kvm_nvhe___vgic_v3_read_vmcr = __vgic_v3_read_vmcr; __kvm_nvhe___vgic_v3_restore_aprs = __vgic_v3_restore_aprs; __kvm_nvhe___vgic_v3_save_aprs = __vgic_v3_save_aprs; __kvm_nvhe___vgic_v3_write_vmcr = __vgic_v3_write_vmcr; +__kvm_nvhe_abort_guest_exit_end = abort_guest_exit_end; +__kvm_nvhe_abort_guest_exit_start = abort_guest_exit_start; +__kvm_nvhe_arm64_enable_wa2_handling = arm64_enable_wa2_handling; +__kvm_nvhe_arm64_ssbd_callback_required = arm64_ssbd_callback_required; +__kvm_nvhe_hyp_panic = hyp_panic; +__kvm_nvhe_kimage_voffset = kimage_voffset; +__kvm_nvhe_kvm_host_data = kvm_host_data; +__kvm_nvhe_kvm_patch_vector_branch = kvm_patch_vector_branch; +__kvm_nvhe_kvm_update_va_mask = kvm_update_va_mask; +__kvm_nvhe_panic = panic;#endif /*
CONFIG_KVM*/arch/arm64/kvm/hyp/hyp-entry.S +2
-0@@ -40,6 +40,7 @@ el1_sync: // Guest trapped into EL2 ccmp x0, #ESR_ELx_EC_HVC32, #4, ne b.ne el1_trap +#ifdef __KVM_NVHE_HYPERVISOR__ mrs x1, vttbr_el2 // If vttbr is valid, the guest cbnz x1, el1_hvc_guest // called HVC@@ -74,6 +75,7 @@ el1_sync: // Guest trapped into EL2 eret sb +#endif /* __KVM_NVHE_HYPERVISOR__ */ el1_hvc_guest: /*arch/arm64/kvm/hyp/nvhe/Makefile +1
-1@@ -7,7 +7,7 @@ asflags-y := -D__KVM_NVHE_HYPERVISOR__ ccflags-y := -D__KVM_NVHE_HYPERVISOR__ -fno-stack-protector \ -DDISABLE_BRANCH_PROFILING $(DISABLE_STACKLEAK_PLUGIN) -obj-y := +obj-y := ../hyp-entry.o obj-y := $(patsubst %.o,%.hyp.o,$(obj-y)) extra-y := $(patsubst %.hyp.o,%.hyp.tmp.o,$(obj-y))arch/arm64/kvm/va_layout.c +0
-0
arch/arm64/kvm/va_layout.c void kvm_patch_vector_branch(struct alt_instr *alt, · line 155
Quoted 7 lines
arch/arm64/kvm/va_layout.c +1
-1@@ -150,7 +150,7 @@ void kvm_patch_vector_branch(struct alt_instr *alt,/* * Compute HYP VA by using the same computation as kern_hyp_va() */ - addr = (uintptr_t )kvm_ksym_ref (__kvm_hyp_vector); + addr = (uintptr_t )kvm_ksym_ref_nvhe (__kvm_hyp_vector); addr &= va_mask; addr |= tag_val << tag_lsb;
Thanks,
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
Re: [PATCH v3 05/15] arm64: kvm: Build hyp-entry.S separately for VHE/nVHE
Hi Marc,
Quoted 6 lines
Review
Quoted 3 lines
(quoted diff) +2
-1- void *dst = lm_alias(__bp_harden_hyp_vecs + slot * SZ_2K); + char *vec = has_vhe() ? __bp_harden_hyp_vecs + : kvm_nvhe_sym(__bp_harden_hyp_vecs);If we get this construct often, then something that abstracts the uggliness of the symbol duality would be nice...
Agreed, I do hope that this will end up being limited to finding the address of the hyp-init vector once EL2 becomes self-contained. Even this vector selection can be done in EL2 where the symbol duality does not exist.
If we were to hide it, there is a trade off between code "elegance" and clarity of what's happening under the hood. I was thinking we could extract this has_vhe() ? foo : __kvm_nvhe_foo as a #define foo but I do worry that anybody debugging this code would be cursing my name. It would also not work with other macros that take symbol names, notably kvm_ksym_ref. But that can be rewritten to accept a pointer instead. The more verbose but less magic approach is to have a bunch of different helpers for various situations, eg. __pa_symbol_nvhe. What would be your preference?
-David
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Marc Zyngier
maintainerRe: [PATCH v3 05/15] arm64: kvm: Build hyp-entry.S separately for VHE/nVHE
Hi David,
Quoted 9 lines · David Brazdil
Hi Marc,
Quoted 6 lines
Review
Quoted 3 lines
(quoted diff) +2
-1- void *dst = lm_alias(__bp_harden_hyp_vecs + slot * SZ_2K); + char *vec = has_vhe() ? __bp_harden_hyp_vecs + : kvm_nvhe_sym(__bp_harden_hyp_vecs);If we get this construct often, then something that abstracts the uggliness of the symbol duality would be nice...
Quoted 7 lines
Agreed, I do hope that this will end up being limited to finding the address of the hyp-init vector once EL2 becomes self-contained. Even this vector selection can be done in EL2 where the symbol duality does not exist.
If we were to hide it, there is a trade off between code "elegance" and clarity of what's happening under the hood. I was thinking we could extract thishas_vhe() ? foo : __kvm_nvhe_fooas a#define foobut I do worry that anybody debugging this code would be cursing my name.
I would say that whoever is debugging this code better have an understanding of how things are supposed to work. Given that this is only a handful of people so far, I think your name is safe! ;-)
Quoted 5 lines
It would also not work
with other macros that take symbol names, notablykvm_ksym_ref. But that can be rewritten to accept a pointer instead. The more verbose but less magic approach is to have a bunch of different helpers for various situations, eg.__pa_symbol_nvhe. What would be your preference?
I'd be happy with the (maybe temporary) magic approach. It helps reasoning about things, and makes the transition smoother. Yes, bugs could crop up there, but given the static nature of obtaining a symbol's address, I'm fairly confident we'll get it right. The same cannot be said about pointers though.
Thanks,
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
Re: [PATCH v3 05/15] arm64: kvm: Build hyp-entry.S separately for VHE/nVHE
Hey Marc,
Quoted 5 lines
I'd be happy with the (maybe temporary) magic approach. It helps reasoning about things, and makes the transition smoother. Yes, bugs could crop up there, but given the static nature of obtaining a symbol's address, I'm fairly confident we'll get it right. The same cannot be said about pointers though.
Ok, so this is what it would look like:
arch/arm64/include/asm/kvm_asm.h +12 -4
@@ -59,13 +59,14 @@#define DECLARE_KVM_HYP_SYM (sym) \ DECLARE_KVM_VHE_SYM(sym); \ DECLARE_KVM_NVHE_SYM(sym) +#define CHOOSE_HYP_SYM (sym) (has_vhe() ? sym : kvm_nvhe_sym(sym))/* Translate a kernel address of @sym into its equivalent linear mapping */ -#define kvm_ksym_ref (sym) \ +#define kvm_ksym_ref (ptr) \ ({ \ - void *val = &sym; \ + void *val = (ptr); \ if (!is_kernel_in_hyp_mode()) \ - val = lm_alias(&sym); \ + val = lm_alias((ptr)); \ val; \ })#define kvm_ksym_ref_nvhe (sym) kvm_ksym_ref(kvm_nvhe_sym(sym))
@@ -76,7 +77,14 @@ struct kvm_vcpu;extern char __kvm_hyp_init[];extern char __kvm_hyp_init_end[]; -extern char __kvm_hyp_vector[]; +DECLARE_KVM_HYP_SYM (__kvm_hyp_vector); +#define __kvm_hyp_vector CHOOSE_HYP_SYM(__kvm_hyp_vector) + +#ifdef CONFIG_KVM_INDIRECT_VECTORS +DECLARE_KVM_HYP_SYM (__bp_harden_hyp_vecs); +#define __bp_harden_hyp_vecs CHOOSE_HYP_SYM(__bp_harden_hyp_vecs) +extern atomic_t arm64_el2_vector_last_slot; +#endif extern void __kvm_flush_vm_context (void );extern void __kvm_tlb_flush_vmid_ipa (struct kvm *kvm,phys_addr_t ipa);
Everything compiles and boots. Only existing code I had to change was:
arch/arm64/kvm/arm.c +1 -1
@@ -1285,7 +1285,7 @@ static void cpu_init_hyp_mode(void)* so that we can use adr_l to access per-cpu variables in EL2. */ tpidr_el2 = ((unsigned long )this_cpu_ptr (&kvm_host_data) - - (unsigned long )kvm_ksym_ref (kvm_host_data)); + (unsigned long )kvm_ksym_ref (&kvm_host_data)); pgd_ptr =kvm_mmu_get_httbr (); hyp_stack_ptr =__this_cpu_read (kvm_arm_hyp_stack_page) +PAGE_SIZE ;
WDYT?
Thanks,
-David
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v3 6/15 arm64: kvm: Move hyp-init.S to nVHE
In reply to David BrazdilFrom: Andrew Scull <[email protected]>
This patch is part of a series which builds KVM's non-VHE hyp code separately from VHE and the rest of the kernel.
hyp-init.S contains the identity mapped initialisation code for the non-VHE code that runs at EL2. It is only used for non-VHE.
Adjust code that calls into this to use the prefixed symbol name.
---
arch/arm64/include/asm/kvm_asm.h | 4 +---
arch/arm64/kernel/image-vars.h | 3 ++-
arch/arm64/kvm/Makefile | 2 +-
arch/arm64/kvm/hyp/nvhe/Makefile | 2 +-
arch/arm64/kvm/{ => hyp/nvhe}/hyp-init.S | 0
arch/arm64/kvm/mmu.c | 2 +-
6 files changed, 6 insertions(+), 7 deletions(-)
rename arch/arm64/kvm/{ => hyp/nvhe}/hyp-init.S (100%)arch/arm64/include/asm/kvm_asm.h +1 -3
@@ -73,9 +73,7 @@struct kvm ;struct kvm_vcpu ; -extern char __kvm_hyp_init[]; -extern char __kvm_hyp_init_end[]; - +DECLARE_KVM_NVHE_SYM (__kvm_hyp_init);DECLARE_KVM_HYP_SYM (__kvm_hyp_vector);#ifdef CONFIG_KVM_INDIRECT_VECTORS
arch/arm64/kernel/image-vars.h +2 -1
@@ -62,10 +62,10 @@ __efistub__ctype = _ctype;*/ __kvm_nvhe___guest_exit = __guest_exit; +__kvm_nvhe___hyp_stub_vectors = __hyp_stub_vectors; __kvm_nvhe___kvm_enable_ssbs = __kvm_enable_ssbs; __kvm_nvhe___kvm_flush_vm_context = __kvm_flush_vm_context; __kvm_nvhe___kvm_get_mdcr_el2 = __kvm_get_mdcr_el2; -__kvm_nvhe___kvm_handle_stub_hvc = __kvm_handle_stub_hvc; __kvm_nvhe___kvm_timer_set_cntvoff = __kvm_timer_set_cntvoff; __kvm_nvhe___kvm_tlb_flush_local_vmid = __kvm_tlb_flush_local_vmid; __kvm_nvhe___kvm_tlb_flush_vmid = __kvm_tlb_flush_vmid;
@@ -82,6 +82,7 @@ __kvm_nvhe_abort_guest_exit_start = abort_guest_exit_start; __kvm_nvhe_arm64_enable_wa2_handling = arm64_enable_wa2_handling; __kvm_nvhe_arm64_ssbd_callback_required = arm64_ssbd_callback_required; __kvm_nvhe_hyp_panic = hyp_panic; +__kvm_nvhe_idmap_t0sz = idmap_t0sz; __kvm_nvhe_kimage_voffset = kimage_voffset; __kvm_nvhe_kvm_host_data = kvm_host_data; __kvm_nvhe_kvm_patch_vector_branch = kvm_patch_vector_branch;
arch/arm64/kvm/Makefile +1 -1
@@ -13,7 +13,7 @@ obj-$(CONFIG_KVM) += hyp/ kvm-y := $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o \ $(KVM)/vfio.o $(KVM)/irqchip.o \ arm.o mmu.o mmio.o psci.o perf.o hypercalls.o pvtime.o \ - inject_fault.o regmap.o va_layout.o hyp.o hyp-init.o handle_exit.o \ + inject_fault.o regmap.o va_layout.o hyp.o handle_exit.o \ guest.o debug.o reset.o sys_regs.o sys_regs_generic_v8.o \ vgic-sys-reg-v3.o fpsimd.o pmu.o \ aarch32.o arch_timer.o \
arch/arm64/kvm/hyp/nvhe/Makefile +1 -1
@@ -7,7 +7,7 @@ asflags-y := -D__KVM_NVHE_HYPERVISOR__ ccflags-y := -D__KVM_NVHE_HYPERVISOR__ -fno-stack-protector \ -DDISABLE_BRANCH_PROFILING $(DISABLE_STACKLEAK_PLUGIN) -obj-y := ../hyp-entry.o +obj-y := hyp-init.o ../hyp-entry.o obj-y := $(patsubst %.o,%.hyp.o,$(obj-y)) extra-y := $(patsubst %.hyp.o,%.hyp.tmp.o,$(obj-y))
arch/arm64/kvm/hyp/nvhe/hyp-init.S renamed from arch/arm64/kvm/hyp-init.S +0 -0
arch/arm64/kvm/mmu.c +1 -1
@@ -2346,7 +2346,7 @@ int kvm_mmu_init(void) hyp_idmap_start =ALIGN_DOWN (hyp_idmap_start,PAGE_SIZE ); hyp_idmap_end =__pa_symbol (__hyp_idmap_text_end); hyp_idmap_end =ALIGN (hyp_idmap_end,PAGE_SIZE ); - hyp_idmap_vector =__pa_symbol (__kvm_hyp_init); + hyp_idmap_vector =__pa_symbol (kvm_nvhe_sym (__kvm_hyp_init));/* * We rely on the linker script to ensure at build time that the HYP
Signature · 6 lines
2.27.0 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v3 7/15 arm64: kvm: Split hyp/tlb.c to VHE/nVHE
In reply to David BrazdilThis patch is part of a series which builds KVM's non-VHE hyp code separately from VHE and the rest of the kernel.
tlb.c contains code for flushing the TLB, with parts shared between VHE/nVHE. These common routines are moved into a header file tlb.h, VHE-specific code remains in tlb.c and nVHE-specific code is moved to nvhe/tlb.c.
The header file expects its users to implement two helper functions declared at the top of the file.
--- arch/arm64/kernel/image-vars.h | 8 +- arch/arm64/kvm/hyp/nvhe/Makefile | 2 +- arch/arm64/kvm/hyp/nvhe/tlb.c | 70 +++++++++++++ arch/arm64/kvm/hyp/tlb.c | 171 +++---------------------------- arch/arm64/kvm/hyp/tlb.h | 134 ++++++++++++++++++++++++ 5 files changed, 222 insertions(+), 163 deletions(-) create mode 100644 arch/arm64/kvm/hyp/nvhe/tlb.c create mode 100644 arch/arm64/kvm/hyp/tlb.h
arch/arm64/kernel/image-vars.h +4 -4
@@ -63,13 +63,10 @@ __efistub__ctype = _ctype; __kvm_nvhe___guest_exit = __guest_exit; __kvm_nvhe___hyp_stub_vectors = __hyp_stub_vectors; +__kvm_nvhe___icache_flags = __icache_flags; __kvm_nvhe___kvm_enable_ssbs = __kvm_enable_ssbs; -__kvm_nvhe___kvm_flush_vm_context = __kvm_flush_vm_context; __kvm_nvhe___kvm_get_mdcr_el2 = __kvm_get_mdcr_el2; __kvm_nvhe___kvm_timer_set_cntvoff = __kvm_timer_set_cntvoff; -__kvm_nvhe___kvm_tlb_flush_local_vmid = __kvm_tlb_flush_local_vmid; -__kvm_nvhe___kvm_tlb_flush_vmid = __kvm_tlb_flush_vmid; -__kvm_nvhe___kvm_tlb_flush_vmid_ipa = __kvm_tlb_flush_vmid_ipa; __kvm_nvhe___kvm_vcpu_run_nvhe = __kvm_vcpu_run_nvhe; __kvm_nvhe___vgic_v3_get_ich_vtr_el2 = __vgic_v3_get_ich_vtr_el2; __kvm_nvhe___vgic_v3_init_lrs = __vgic_v3_init_lrs;
@@ -79,8 +76,11 @@ __kvm_nvhe___vgic_v3_save_aprs = __vgic_v3_save_aprs; __kvm_nvhe___vgic_v3_write_vmcr = __vgic_v3_write_vmcr; __kvm_nvhe_abort_guest_exit_end = abort_guest_exit_end; __kvm_nvhe_abort_guest_exit_start = abort_guest_exit_start; +__kvm_nvhe_arm64_const_caps_ready = arm64_const_caps_ready; __kvm_nvhe_arm64_enable_wa2_handling = arm64_enable_wa2_handling; __kvm_nvhe_arm64_ssbd_callback_required = arm64_ssbd_callback_required; +__kvm_nvhe_cpu_hwcap_keys = cpu_hwcap_keys; +__kvm_nvhe_cpu_hwcaps = cpu_hwcaps; __kvm_nvhe_hyp_panic = hyp_panic; __kvm_nvhe_idmap_t0sz = idmap_t0sz; __kvm_nvhe_kimage_voffset = kimage_voffset;
arch/arm64/kvm/hyp/nvhe/Makefile +1 -1
@@ -7,7 +7,7 @@ asflags-y := -D__KVM_NVHE_HYPERVISOR__ ccflags-y := -D__KVM_NVHE_HYPERVISOR__ -fno-stack-protector \ -DDISABLE_BRANCH_PROFILING $(DISABLE_STACKLEAK_PLUGIN) -obj-y := hyp-init.o ../hyp-entry.o +obj-y := tlb.o hyp-init.o ../hyp-entry.o obj-y := $(patsubst %.o,%.hyp.o,$(obj-y)) extra-y := $(patsubst %.hyp.o,%.hyp.tmp.o,$(obj-y))
arch/arm64/kvm/hyp/nvhe/tlb.c new file +70 -0
70 lines not on this page · raw message
arch/arm64/kvm/hyp/tlb.c +13 -158
198 lines not on this page · raw message
arch/arm64/kvm/hyp/tlb.h new file +134 -0
134 lines not on this page · raw message
Signature · 6 lines
2.27.0 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Marc Zyngier
maintainerRe: [PATCH v3 07/15] arm64: kvm: Split hyp/tlb.c to VHE/nVHE
Quoted 26 lines · David Brazdil
This patch is part of a series which builds KVM's non-VHE hyp code separately from VHE and the rest of the kernel.
tlb.ccontains code for flushing the TLB, with parts shared between VHE/nVHE. These common routines are moved into a header filetlb.h, VHE-specific code remains intlb.cand nVHE-specific code is moved tonvhe/tlb.c.The header file expects its users to implement two helper functions declared at the top of the file.
Signed-off-by David Brazdil <[email protected]>--- arch/arm64/kernel/image-vars.h | 8 +- arch/arm64/kvm/hyp/nvhe/Makefile | 2 +- arch/arm64/kvm/hyp/nvhe/tlb.c | 70 +++++++++++++ arch/arm64/kvm/hyp/tlb.c | 171 +++---------------------------- arch/arm64/kvm/hyp/tlb.h | 134 ++++++++++++++++++++++++ 5 files changed, 222 insertions(+), 163 deletions(-) create mode 100644 arch/arm64/kvm/hyp/nvhe/tlb.c create mode 100644 arch/arm64/kvm/hyp/tlb.harch/arm64/kernel/image-vars.h +0
-0
arch/arm64/kernel/image-vars.h __efistub__ctype = _ctype; · line 66
Quoted 4 lines
arch/arm64/kernel/image-vars.h +1
-0@@ -63,13 +63,10 @@ __efistub__ctype = _ctype; __kvm_nvhe___guest_exit = __guest_exit; __kvm_nvhe___hyp_stub_vectors = __hyp_stub_vectors; +__kvm_nvhe___icache_flags = __icache_flags;
This is new, and definitely deserves a comment, together with much of the other kernel symbols you added. You probably want to keep these symbols separate from the KVM-specific symbols for ease of maintenance.
Quoted 43 lines
arch/arm64/kernel/image-vars.h +3
-4__kvm_nvhe___kvm_enable_ssbs = __kvm_enable_ssbs; -__kvm_nvhe___kvm_flush_vm_context = __kvm_flush_vm_context; __kvm_nvhe___kvm_get_mdcr_el2 = __kvm_get_mdcr_el2; __kvm_nvhe___kvm_timer_set_cntvoff = __kvm_timer_set_cntvoff; -__kvm_nvhe___kvm_tlb_flush_local_vmid = __kvm_tlb_flush_local_vmid; -__kvm_nvhe___kvm_tlb_flush_vmid = __kvm_tlb_flush_vmid; -__kvm_nvhe___kvm_tlb_flush_vmid_ipa = __kvm_tlb_flush_vmid_ipa; __kvm_nvhe___kvm_vcpu_run_nvhe = __kvm_vcpu_run_nvhe; __kvm_nvhe___vgic_v3_get_ich_vtr_el2 = __vgic_v3_get_ich_vtr_el2; __kvm_nvhe___vgic_v3_init_lrs = __vgic_v3_init_lrs;@@ -79,8 +76,11 @@ __kvm_nvhe___vgic_v3_save_aprs = __vgic_v3_save_aprs; __kvm_nvhe___vgic_v3_write_vmcr = __vgic_v3_write_vmcr; __kvm_nvhe_abort_guest_exit_end = abort_guest_exit_end; __kvm_nvhe_abort_guest_exit_start = abort_guest_exit_start; +__kvm_nvhe_arm64_const_caps_ready = arm64_const_caps_ready; __kvm_nvhe_arm64_enable_wa2_handling = arm64_enable_wa2_handling; __kvm_nvhe_arm64_ssbd_callback_required = arm64_ssbd_callback_required; +__kvm_nvhe_cpu_hwcap_keys = cpu_hwcap_keys; +__kvm_nvhe_cpu_hwcaps = cpu_hwcaps; __kvm_nvhe_hyp_panic = hyp_panic;
__kvm_nvhe_idmap_t0sz=idmap_t0sz;
__kvm_nvhe_kimage_voffset=kimage_voffset;arch/arm64/kvm/hyp/nvhe/Makefile +1
-1@@ -7,7 +7,7 @@ asflags-y := -D__KVM_NVHE_HYPERVISOR__ ccflags-y := -D__KVM_NVHE_HYPERVISOR__ -fno-stack-protector \ -DDISABLE_BRANCH_PROFILING $(DISABLE_STACKLEAK_PLUGIN) -obj-y := hyp-init.o ../hyp-entry.o +obj-y := tlb.o hyp-init.o ../hyp-entry.o obj-y := $(patsubst %.o,%.hyp.o,$(obj-y)) extra-y := $(patsubst %.hyp.o,%.hyp.tmp.o,$(obj-y))arch/arm64/kvm/hyp/nvhe/tlb.c new file +0
-0
arch/arm64/kvm/hyp/nvhe/tlb.c line 7
Quoted 7 lines
arch/arm64/kvm/hyp/nvhe/tlb.c new file +7
-0@@ -0,0 +1,70 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Copyright (C) 2015 - ARM Ltd +* Author: Marc Zyngier <[email protected]> +*/ + +#include <linux/irqflags.h>
Unnecessary non !VHE.
arch/arm64/kvm/hyp/nvhe/tlb.c line 11
Quoted 4 lines
arch/arm64/kvm/hyp/nvhe/tlb.c new file +4
-0+ +#include <asm/kvm_hyp.h> +#include <asm/kvm_mmu.h> +#include <asm/tlbflush.h>
At least tlbflush.h should directly be dragged by tlb.h, since that's where the actual TLB ops are done.
Quoted 50 lines
arch/arm64/kvm/hyp/nvhe/tlb.c new file +41
-0+ +#include "../tlb.h" + +static void __hyp_text __tlb_switch_to_guest(struct kvm *kvm, +struct tlb_inv_context *cxt) +{ +if (cpus_have_final_cap (ARM64_WORKAROUND_SPECULATIVE_AT )) { +u64 val; + +/* +* For CPUs that are affected by ARM 1319367, we need to +* avoid a host Stage-1 walk while we have the guest's +* VMID set in the VTTBR in order to invalidate TLBs. +* We're guaranteed that the S1 MMU is enabled, so we can +* simply set the EPD bits to avoid any further TLB fill. +*/ + val = cxt->tcr =read_sysreg_el1 (SYS_TCR ); + val |=TCR_EPD1_MASK |TCR_EPD0_MASK ; +write_sysreg_el1 (val,SYS_TCR ); +isb (); + } + +/* __load_guest_stage2() includes an ISB for the workaround. */ +__load_guest_stage2 (kvm); + asm(ALTERNATIVE("isb", "nop", ARM64_WORKAROUND_SPECULATIVE_AT)); +} + +static void __hyp_text __tlb_switch_to_host(struct kvm *kvm, +struct tlb_inv_context *cxt) +{ +write_sysreg (0 , vttbr_el2); + +if (cpus_have_final_cap (ARM64_WORKAROUND_SPECULATIVE_AT )) { +/* Ensure write of the host VMID */ +isb (); +/* Restore the host's TCR_EL1 */ +write_sysreg_el1 (cxt->tcr,SYS_TCR ); + } +} + +void __hyp_text __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_tipa)
arch/arm64/kvm/hyp/nvhe/tlb.c new file +8
-0+{ +__tlb_flush_vmid_ipa (kvm, ipa); +} + +void __hyp_text __kvm_tlb_flush_vmid(struct kvm *kvm) +{ +__tlb_flush_vmid (kvm); +}
arch/arm64/kvm/hyp/nvhe/tlb.c line 70
Quoted 10 lines
arch/arm64/kvm/hyp/nvhe/tlb.c new file +10
-0+ +void __hyp_text __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu) +{ +__tlb_flush_local_vmid (vcpu); +} + +void __hyp_text __kvm_flush_vm_context(void) +{ +__tlb_flush_vm_context (); +}
Overall, I find the result hard to reason about. Too many things happen in the .h file, and reading the .c file feels puzzling (no apparent users of the two static functions, for example).
The VHE/nVHE files only differ by the the __tlb_switch_to_*() helpers (and the __hyp_text annotation that ends up going away). Why can't this be structured in a more conventionnal way, where the TLB code stays in a C file, and per-mode .h files providing the two helpers? It would certainly look much more readable.
Quoted 232 lines
arch/arm64/kvm/hyp/tlb.c +5
-42@@ -10,14 +10,10 @@#include <asm/kvm_mmu.h> #include <asm/tlbflush.h> -struct tlb_inv_context { -unsigned long flags; -u64 tcr; -u64 sctlr; -}; +#include "tlb.h" -static void __hyp_text __tlb_switch_to_guest_vhe(struct kvm *kvm, -struct tlb_inv_context *cxt) +static void __hyp_text __tlb_switch_to_guest(struct kvm *kvm, +struct tlb_inv_context *cxt) {u64 val;@@ -60,41 +56,8 @@ static void __hyp_text __tlb_switch_to_guest_vhe(struct kvm *kvm,isb (); } -static void __hyp_text __tlb_switch_to_guest_nvhe(struct kvm *kvm, -struct tlb_inv_context *cxt) -{ -if (cpus_have_final_cap (ARM64_WORKAROUND_SPECULATIVE_AT )) { -u64 val; - -/* -* For CPUs that are affected by ARM 1319367, we need to -* avoid a host Stage-1 walk while we have the guest's -* VMID set in the VTTBR in order to invalidate TLBs. -* We're guaranteed that the S1 MMU is enabled, so we can -* simply set the EPD bits to avoid any further TLB fill. -*/ - val = cxt->tcr =read_sysreg_el1 (SYS_TCR ); - val |=TCR_EPD1_MASK |TCR_EPD0_MASK ; -write_sysreg_el1 (val,SYS_TCR ); -isb (); - } - -/* __load_guest_stage2() includes an ISB for the workaround. */ -__load_guest_stage2 (kvm); - asm(ALTERNATIVE("isb", "nop", ARM64_WORKAROUND_SPECULATIVE_AT)); -} - -static void __hyp_text __tlb_switch_to_guest(struct kvm *kvm, -struct tlb_inv_context *cxt) -{ -if (has_vhe ()) -__tlb_switch_to_guest_vhe (kvm, cxt); -else -__tlb_switch_to_guest_nvhe (kvm, cxt); -} - -static void __hyp_text __tlb_switch_to_host_vhe(struct kvm *kvm, -struct tlb_inv_context *cxt) +static void __hyp_text __tlb_switch_to_host(struct kvm *kvm, +struct tlb_inv_context *cxt) {/* arch/arm64/kvm/hyp/tlb.c +8
-116* We're done with the TLB operation, let's restore the host's @@ -113,130 +76,22 @@ static void __hyp_text __tlb_switch_to_host_vhe(struct kvm *kvm, local_irq_restore(cxt->flags); } -static void __hyp_text __tlb_switch_to_host_nvhe(struct kvm *kvm, -struct tlb_inv_context *cxt) -{ -write_sysreg (0 , vttbr_el2); - -if (cpus_have_final_cap (ARM64_WORKAROUND_SPECULATIVE_AT )) { -/* Ensure write of the host VMID */ -isb (); -/* Restore the host's TCR_EL1 */ -write_sysreg_el1 (cxt->tcr,SYS_TCR ); - } -} - -static void __hyp_text __tlb_switch_to_host(struct kvm *kvm, -struct tlb_inv_context *cxt) -{ -if (has_vhe ()) -__tlb_switch_to_host_vhe (kvm, cxt); -else -__tlb_switch_to_host_nvhe (kvm, cxt); -} - -void __hyp_text __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa) +void __kvm_tlb_flush_vmid_ipa (struct kvm *kvm,phys_addr_t ipa) { -struct tlb_inv_context cxt; - -dsb (ishst); - -/* Switch to requested VMID */ - kvm =kern_hyp_va (kvm); -__tlb_switch_to_guest (kvm, &cxt); - -/* -* We could do so much better if we had the VA as well. -* Instead, we invalidate Stage-2 for this IPA, and the -* whole of Stage-1. Weep... -*/ - ipa >>=12 ; -__tlbi (ipas2e1is, ipa); - -/* -* We have to ensure completion of the invalidation at Stage-2, -* since a table walk on another CPU could refill a TLB with a -* complete (S1 + S2) walk based on the old Stage-2 mapping if -* the Stage-1 invalidation happened first. -*/ -dsb (ish); -__tlbi (vmalle1is); -dsb (ish); -isb (); - -/* -* If the host is running at EL1 and we have a VPIPT I-cache, -* then we must perform I-cache maintenance at EL2 in order for -* it to have an effect on the guest. Since the guest cannot hit -* I-cache lines allocated with a different VMID, we don't need -* to worry about junk out of guest reset (we nuke the I-cache on -* VMID rollover), but we do need to be careful when remapping -* executable pages for the same guest. This can happen when KSM -* takes a CoW fault on an executable page, copies the page into -* a page that was previously mapped in the guest and then needs -* to invalidate the guest view of the I-cache for that page -* from EL1. To solve this, we invalidate the entire I-cache when -* unmapping a page from a guest if we have a VPIPT I-cache but -* the host is running at EL1. As above, we could do better if -* we had the VA. -* -* The moral of this story is: if you have a VPIPT I-cache, then -* you should be running with VHE enabled. -*/ -if (!has_vhe () &&icache_is_vpipt ()) -__flush_icache_all (); - -__tlb_switch_to_host (kvm, &cxt); +__tlb_flush_vmid_ipa (kvm, ipa); } -void __hyp_text __kvm_tlb_flush_vmid(struct kvm *kvm) +void __kvm_tlb_flush_vmid (struct kvm *kvm) { -struct tlb_inv_context cxt; - -dsb (ishst); - -/* Switch to requested VMID */ - kvm =kern_hyp_va (kvm); -__tlb_switch_to_guest (kvm, &cxt); - -__tlbi (vmalls12e1is); -dsb (ish); -isb (); - -__tlb_switch_to_host (kvm, &cxt); +__tlb_flush_vmid (kvm); } -void __hyp_text __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu) +void __kvm_tlb_flush_local_vmid (struct kvm_vcpu *vcpu) { -struct kvm *kvm =kern_hyp_va (kern_hyp_va (vcpu)->kvm); -struct tlb_inv_context cxt; - -/* Switch to requested VMID */ -__tlb_switch_to_guest (kvm, &cxt); - -__tlbi (vmalle1); -dsb (nsh); -isb (); - -__tlb_switch_to_host (kvm, &cxt); +__tlb_flush_local_vmid (vcpu); } -void __hyp_text __kvm_flush_vm_context(void) +void __kvm_flush_vm_context (void ) { -dsb (ishst); -__tlbi (alle1is); - -/* -* VIPT and PIPT caches are not affected by VMID, so no maintenance -* is necessary across a VMID rollover. -* -* VPIPT caches constrain lookup and maintenance to the active VMID, -* so we need to invalidate lines with a stale VMID to avoid an ABA -* race after multiple rollovers. -* -*/ -if (icache_is_vpipt ()) - asmvolatile ("ic ialluis" ); - -dsb (ish); +__tlb_flush_vm_context (); }arch/arm64/kvm/hyp/tlb.h new file +18
-0@@ -0,0 +1,134 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Copyright (C) 2015 - ARM Ltd +* Author: Marc Zyngier <[email protected]> +*/ + +#ifndef __ARM64_KVM_HYP_TLB_H__ +#define __ARM64_KVM_HYP_TLB_H__ + +#include <linux/irqflags.h> + +#include <asm/kvm_hyp.h> +#include <asm/kvm_mmu.h> +#include <asm/tlbflush.h> + +struct tlb_inv_context { + unsigned long flags; + u64 tcr;
arch/arm64/kvm/hyp/tlb.h line 28
Quoted 10 lines
arch/arm64/kvm/hyp/tlb.h new file +10
-0+u64 sctlr; +}; + +static void __hyp_text __tlb_switch_to_guest(struct kvm *kvm, +struct tlb_inv_context *cxt); +static void __hyp_text __tlb_switch_to_host(struct kvm *kvm, +struct tlb_inv_context *cxt); + +static inline void __hyp_text +__tlb_flush_vmid_ipa (struct kvm *kvm,phys_addr_t ipa)
For things that you move around, please do not reformat the code, This will lead to unnecessary conflicts. And long lines are just fine (screw checkpatch!).
Thanks,
M.
Signature · 5 lines
Jazz is not dead. It just smells funny... _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v3 8/15 arm64: kvm: Split hyp/switch.c to VHE/nVHE
In reply to David BrazdilThis patch is part of a series which builds KVM's non-VHE hyp code separately from VHE and the rest of the kernel.
switch.c implements context-switching for KVM, with large parts shared between VHE/nVHE. These common routines are moved to switch.h, VHE-specific code is left in switch.c and nVHE-specific code is moved to nvhe/switch.c.
Previously __kvm_vcpu_run needed a different symbol name for VHE/nVHE. This is cleaned up and the caller in arm.c simplified.
--- arch/arm64/include/asm/kvm_asm.h | 4 +- arch/arm64/include/asm/kvm_hyp.h | 5 + arch/arm64/kernel/image-vars.h | 31 +- arch/arm64/kvm/arm.c | 6 +- arch/arm64/kvm/hyp/hyp-entry.S | 2 + arch/arm64/kvm/hyp/nvhe/Makefile | 2 +- arch/arm64/kvm/hyp/nvhe/switch.c | 271 +++++++++++ arch/arm64/kvm/hyp/switch.c | 749 +------------------------------ arch/arm64/kvm/hyp/switch.h | 507 +++++++++++++++++++++ arch/arm64/kvm/hyp/sysreg-sr.c | 4 +- 10 files changed, 835 insertions(+), 746 deletions(-) create mode 100644 arch/arm64/kvm/hyp/nvhe/switch.c create mode 100644 arch/arm64/kvm/hyp/switch.h
arch/arm64/include/asm/kvm_asm.h +1 -3
@@ -88,9 +88,7 @@ extern void __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu);extern void __kvm_timer_set_cntvoff (u64 cntvoff); -extern int kvm_vcpu_run_vhe (struct kvm_vcpu *vcpu); - -extern int __kvm_vcpu_run_nvhe (struct kvm_vcpu *vcpu); +extern int __kvm_vcpu_run (struct kvm_vcpu *vcpu);extern void __kvm_enable_ssbs (void );
arch/arm64/include/asm/kvm_hyp.h +5 -0
@@ -81,11 +81,16 @@ void __debug_switch_to_host(struct kvm_vcpu *vcpu);void __fpsimd_save_state (struct user_fpsimd_state *fp_regs);void __fpsimd_restore_state (struct user_fpsimd_state *fp_regs); +#ifndef __KVM_NVHE_HYPERVISOR__void activate_traps_vhe_load (struct kvm_vcpu *vcpu);void deactivate_traps_vhe_put (void ); +#endif u64 __guest_enter (struct kvm_vcpu *vcpu,struct kvm_cpu_context *host_ctxt); + +#ifdef __KVM_NVHE_HYPERVISOR__void __noreturn __hyp_do_panic (unsigned long , ...); +#endif #endif /* __ARM64_KVM_HYP_H__ */
arch/arm64/kernel/image-vars.h +29 -2
@@ -61,18 +61,35 @@ __efistub__ctype = _ctype;* memory mappings. */ +__kvm_nvhe___debug_switch_to_guest = __debug_switch_to_guest; +__kvm_nvhe___debug_switch_to_host = __debug_switch_to_host; +__kvm_nvhe___fpsimd_restore_state = __fpsimd_restore_state; +__kvm_nvhe___fpsimd_save_state = __fpsimd_save_state; +__kvm_nvhe___guest_enter = __guest_enter; __kvm_nvhe___guest_exit = __guest_exit; +__kvm_nvhe___hyp_panic_string = __hyp_panic_string; __kvm_nvhe___hyp_stub_vectors = __hyp_stub_vectors; __kvm_nvhe___icache_flags = __icache_flags; __kvm_nvhe___kvm_enable_ssbs = __kvm_enable_ssbs; __kvm_nvhe___kvm_get_mdcr_el2 = __kvm_get_mdcr_el2; __kvm_nvhe___kvm_timer_set_cntvoff = __kvm_timer_set_cntvoff; -__kvm_nvhe___kvm_vcpu_run_nvhe = __kvm_vcpu_run_nvhe; +__kvm_nvhe___sysreg32_restore_state = __sysreg32_restore_state; +__kvm_nvhe___sysreg32_save_state = __sysreg32_save_state; +__kvm_nvhe___sysreg_restore_state_nvhe = __sysreg_restore_state_nvhe; +__kvm_nvhe___sysreg_save_state_nvhe = __sysreg_save_state_nvhe; +__kvm_nvhe___timer_disable_traps = __timer_disable_traps; +__kvm_nvhe___timer_enable_traps = __timer_enable_traps; +__kvm_nvhe___vgic_v2_perform_cpuif_access = __vgic_v2_perform_cpuif_access; +__kvm_nvhe___vgic_v3_activate_traps = __vgic_v3_activate_traps; +__kvm_nvhe___vgic_v3_deactivate_traps = __vgic_v3_deactivate_traps; __kvm_nvhe___vgic_v3_get_ich_vtr_el2 = __vgic_v3_get_ich_vtr_el2; __kvm_nvhe___vgic_v3_init_lrs = __vgic_v3_init_lrs; +__kvm_nvhe___vgic_v3_perform_cpuif_access = __vgic_v3_perform_cpuif_access; __kvm_nvhe___vgic_v3_read_vmcr = __vgic_v3_read_vmcr; __kvm_nvhe___vgic_v3_restore_aprs = __vgic_v3_restore_aprs; +__kvm_nvhe___vgic_v3_restore_state = __vgic_v3_restore_state; __kvm_nvhe___vgic_v3_save_aprs = __vgic_v3_save_aprs; +__kvm_nvhe___vgic_v3_save_state = __vgic_v3_save_state; __kvm_nvhe___vgic_v3_write_vmcr = __vgic_v3_write_vmcr; __kvm_nvhe_abort_guest_exit_end = abort_guest_exit_end; __kvm_nvhe_abort_guest_exit_start = abort_guest_exit_start;
@@ -81,13 +98,23 @@ __kvm_nvhe_arm64_enable_wa2_handling = arm64_enable_wa2_handling; __kvm_nvhe_arm64_ssbd_callback_required = arm64_ssbd_callback_required; __kvm_nvhe_cpu_hwcap_keys = cpu_hwcap_keys; __kvm_nvhe_cpu_hwcaps = cpu_hwcaps; -__kvm_nvhe_hyp_panic = hyp_panic; +#ifdef CONFIG_ARM64_PSEUDO_NMI +__kvm_nvhe_gic_pmr_sync = gic_pmr_sync; +#endif __kvm_nvhe_idmap_t0sz = idmap_t0sz; __kvm_nvhe_kimage_voffset = kimage_voffset; __kvm_nvhe_kvm_host_data = kvm_host_data; __kvm_nvhe_kvm_patch_vector_branch = kvm_patch_vector_branch; +__kvm_nvhe_kvm_skip_instr32 = kvm_skip_instr32; __kvm_nvhe_kvm_update_va_mask = kvm_update_va_mask; +__kvm_nvhe_kvm_vgic_global_state = kvm_vgic_global_state; __kvm_nvhe_panic = panic; +#ifdef CONFIG_ARM64_SVE +__kvm_nvhe_sve_load_state = sve_load_state; +__kvm_nvhe_sve_save_state = sve_save_state; +#endif +__kvm_nvhe_vgic_v2_cpuif_trap = vgic_v2_cpuif_trap; +__kvm_nvhe_vgic_v3_cpuif_trap = vgic_v3_cpuif_trap;#endif /* CONFIG_KVM */
arch/arm64/kvm/arm.c +1 -5
@@ -748,11 +748,7 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu)trace_kvm_entry (*vcpu_pc (vcpu));guest_enter_irqoff (); -if (has_vhe ()) { - ret =kvm_vcpu_run_vhe (vcpu); - }else { - ret =kvm_call_hyp_ret (__kvm_vcpu_run_nvhe, vcpu); - } + ret =kvm_call_hyp_ret (__kvm_vcpu_run, vcpu); vcpu->mode =OUTSIDE_GUEST_MODE ; vcpu->stat.exits++;
arch/arm64/kvm/hyp/hyp-entry.S +2 -0
@@ -182,6 +182,7 @@ el2_error: eret sb +#ifdef __KVM_NVHE_HYPERVISOR__ SYM_FUNC_START(__hyp_do_panic) mov lr, #(PSR_F_BIT | PSR_I_BIT | PSR_A_BIT | PSR_D_BIT |\ PSR_MODE_EL1h)
@@ -191,6 +192,7 @@ SYM_FUNC_START(__hyp_do_panic) eret sb SYM_FUNC_END(__hyp_do_panic) +#endif SYM_CODE_START(__hyp_panic) get_host_ctxt x0, x1
arch/arm64/kvm/hyp/nvhe/Makefile +1 -1
@@ -7,7 +7,7 @@ asflags-y := -D__KVM_NVHE_HYPERVISOR__ ccflags-y := -D__KVM_NVHE_HYPERVISOR__ -fno-stack-protector \ -DDISABLE_BRANCH_PROFILING $(DISABLE_STACKLEAK_PLUGIN) -obj-y := tlb.o hyp-init.o ../hyp-entry.o +obj-y := switch.o tlb.o hyp-init.o ../hyp-entry.o obj-y := $(patsubst %.o,%.hyp.o,$(obj-y)) extra-y := $(patsubst %.hyp.o,%.hyp.tmp.o,$(obj-y))
arch/arm64/kvm/hyp/nvhe/switch.c new file +271 -0
271 lines not on this page · raw message
arch/arm64/kvm/hyp/switch.c +16 -733
805 lines not on this page · raw message
arch/arm64/kvm/hyp/switch.h new file +507 -0
507 lines not on this page · raw message
arch/arm64/kvm/hyp/sysreg-sr.c +2 -2
@@ -114,7 +114,7 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)/* * Must only be done for guest registers, hence the context * test. We're coming from the host, so SCTLR.M is already -* set. Pairs with __activate_traps_nvhe(). +* set. Pairs with nVHE's __activate_traps(). */ write_sysreg_el1((ctxt->sys_regs[TCR_EL1] | TCR_EPD1_MASK | TCR_EPD0_MASK),
@@ -142,7 +142,7 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt) ctxt->__hyp_running_vcpu) {/* * Must only be done for host registers, hence the context -* test. Pairs with __deactivate_traps_nvhe(). +* test. Pairs with nVHE's __deactivate_traps(). */ isb ();/*
Signature · 6 lines
2.27.0 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
kernel test robot
build botRe: [PATCH v3 08/15] arm64: kvm: Split hyp/switch.c to VHE/nVHE
Thank you for the patch! Perhaps something to improve:
Hi David,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on v5.8-rc1 next-20200618]
[cannot apply to kvmarm/next arm64/for-next/core arm-perf/for-next/perf] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/David-Brazdil/Split-off-nVHE-hyp-code/20200618-203230 base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 1b5044021070efa3259f3e9548dc35d1eb6aa844 config: arm64-allyesconfig (attached as .config) compiler: aarch64-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=arm64
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>, old ones prefixed by <<):
>> arch/arm64/kvm/hyp/nvhe/switch.c:244:28: warning: no previous prototype for 'hyp_panic' [-Wmissing-prototypes]
244 | void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *host_ctxt)
| ^~~~~~~~~
vim +/hyp_panic +244 arch/arm64/kvm/hyp/nvhe/switch.c
243
> 244 void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *host_ctxt)
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
kernel test robot
build botRe: [PATCH v3 08/15] arm64: kvm: Split hyp/switch.c to VHE/nVHE
In reply to David BrazdilThank you for the patch! Perhaps something to improve:
Hi David,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on v5.8-rc2 next-20200624]
[cannot apply to kvmarm/next arm64/for-next/core arm-perf/for-next/perf] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/David-Brazdil/Split-off-nVHE-hyp-code/20200618-203230 base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 1b5044021070efa3259f3e9548dc35d1eb6aa844 config: arm64-randconfig-r021-20200624 (attached as .config) compiler: clang version 11.0.0 (https://github.com/llvm/llvm-project 8911a35180c6777188fefe0954a2451a2b91deaf)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
# install arm64 cross compiling tool for clang build
# apt-get install binutils-aarch64-linux-gnu
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross ARCH=arm64
If you fix the issue, kindly add following tag as appropriate
All warnings (new ones prefixed by >>):
>> arch/arm64/kvm/hyp/nvhe/switch.c:244:28: warning: no previous prototype for function 'hyp_panic' [-Wmissing-prototypes]
void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *host_ctxt)
^ arch/arm64/kvm/hyp/nvhe/switch.c:244:1: note: declare 'static' if the function is not intended to be used outside of this translation unit void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *host_ctxt)
^
static
1 warning generated.
vim +/hyp_panic +244 arch/arm64/kvm/hyp/nvhe/switch.c
243
> 244 void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *host_ctxt)
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all(a)lists.01.org
Marc Zyngier
maintainerRe: [PATCH v3 08/15] arm64: kvm: Split hyp/switch.c to VHE/nVHE
Quoted 32 lines · kernel test robot
Hi David,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on v5.8-rc2 next-20200624]
[cannot apply to kvmarm/next arm64/for-next/core arm-perf/for-next/perf]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use as documented in
https://git-scm.com/docs/git-format-patch]url:
https://github.com/0day-ci/linux/commits/David-Brazdil/Split-off-nVHE-hyp-code/20200618-203230
base:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git1b5044021070efa3259f3e9548dc35d1eb6aa844
config: arm64-randconfig-r021-20200624 (attached as .config)
compiler: clang version 11.0.0 (https://github.com/llvm/llvm-project8911a35180c6777188fefe0954a2451a2b91deaf)
reproduce (this is a W=1 build):
wget
https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross
-O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install arm64 cross compiling tool for clang build # apt-get install binutils-aarch64-linux-gnu
# save the attached .config to linux build tree COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross
ARCH=arm64If you fix the issue, kindly add following tag as appropriate
Quoted 8 lines
Reported-by kernel test robot <[email protected]>All warnings (new ones prefixed by >>):
Quoted 2 lines
Quoted 2 lines
arch/arm64/kvm/hyp/nvhe/switch.c:244:28: warning: no previous prototype for function 'hyp_panic' [-Wmissing-prototypes]void
__hyp_text__noreturnhyp_panic(structkvm_cpu_context*host_ctxt)
I really wish we could turn these warnings off. They don't add much. Or is there an annotation we could stick on the function (something like __called_from_asm_please_leave_me_alone springs to mind...)?
M.
Signature · 1 line
Jazz is not dead. It just smells funny...
Will Deacon
maintainerRe: [PATCH v3 08/15] arm64: kvm: Split hyp/switch.c to VHE/nVHE
Quoted 36 lines · Marc Zyngier
Quoted 35 lines · kernel test robot
Hi David,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on v5.8-rc2 next-20200624]
[cannot apply to kvmarm/next arm64/for-next/core arm-perf/for-next/perf] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use as documented in
https://git-scm.com/docs/git-format-patch]url:
https://github.com/0day-ci/linux/commits/David-Brazdil/Split-off-nVHE-hyp-code/20200618-203230
base:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git1b5044021070efa3259f3e9548dc35d1eb6aa844
config: arm64-randconfig-r021-20200624 (attached as .config)
compiler: clang version 11.0.0 (https://github.com/llvm/llvm-project8911a35180c6777188fefe0954a2451a2b91deaf)
reproduce (this is a W=1 build):
wget
https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross
-O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install arm64 cross compiling tool for clang build # apt-get install binutils-aarch64-linux-gnu
# save the attached .config to linux build tree COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross
ARCH=arm64If you fix the issue, kindly add following tag as appropriate
Reported-by kernel test robot <[email protected]>All warnings (new ones prefixed by >>):
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
arch/arm64/kvm/hyp/nvhe/switch.c:244:28: warning: no previous prototype for function 'hyp_panic' [-Wmissing-prototypes]void
__hyp_text__noreturnhyp_panic(structkvm_cpu_context*host_ctxt)I really wish we could turn these warnings off. They don't add much. Or is there an annotation we could stick on the function (something like
__called_from_asm_please_leave_me_alonesprings to mind...)?
Agreed, I've caught myself skim-reading the kbuild robot reports now because they're often just noise, and then having to force myself to look at them properly when I remember. Even just something in the subject to say "the only problems are W=1 warnings" would help. Is that possible?
Will
Re: [PATCH v3 08/15] arm64: kvm: Split hyp/switch.c to VHE/nVHE
Quoted 43 lines · Will Deacon
Quoted 41 lines · Marc Zyngier
Quoted 39 lines · kernel test robot
Hi David,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on v5.8-rc2 next-20200624]
[cannot apply to kvmarm/next arm64/for-next/core arm-perf/for-next/perf] [If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use as documented in
https://git-scm.com/docs/git-format-patch]url:
https://github.com/0day-ci/linux/commits/David-Brazdil/Split-off-nVHE-hyp-code/20200618-203230
base:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git1b5044021070efa3259f3e9548dc35d1eb6aa844
config: arm64-randconfig-r021-20200624 (attached as .config)
compiler: clang version 11.0.0 (https://github.com/llvm/llvm-project8911a35180c6777188fefe0954a2451a2b91deaf)
reproduce (this is a W=1 build):
wget
https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross
-O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install arm64 cross compiling tool for clang build # apt-get install binutils-aarch64-linux-gnu
# save the attached .config to linux build tree COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross
ARCH=arm64If you fix the issue, kindly add following tag as appropriate
Reported-by kernel test robot <[email protected]>All warnings (new ones prefixed by >>):
Quoted 2 lines
Quoted 2 lines
arch/arm64/kvm/hyp/nvhe/switch.c:244:28: warning: no previous prototype for function 'hyp_panic' [-Wmissing-prototypes]void
__hyp_text__noreturnhyp_panic(structkvm_cpu_context*host_ctxt)
Quoted 8 lines
Quoted 3 lines
I really wish we could turn these warnings off. They don't add much. Or is there an annotation we could stick on the function (something like
__called_from_asm_please_leave_me_alonesprings to mind...)?Agreed, I've caught myself skim-reading the kbuild robot reports now because they're often just noise, and then having to force myself to look at them properly when I remember. Even just something in the subject to say "the only problems are W=1 warnings" would help. Is that possible?
When the W=1 reports started showing up, it took me a while to figure out these warnings were only enabled at W=1. I asked Philip to help denote these in the reports, and Philip was kind enough to add a note in the report about W=1. I agree that the note could still be more prominent. Another part of me wants to move -Wmissing-prototypes to W=2, but that's just biding time until 0day starts reporting on those.
Signature · 2 lines
Thanks, ~Nick Desaulniers
Re: [PATCH v3 08/15] arm64: kvm: Split hyp/switch.c to VHE/nVHE
In reply to Marc ZyngierQuoted 4 lines · Marc Zyngier
I really wish we could turn these warnings off. They don't add much. Or is there an annotation we could stick on the function (something like
__called_from_asm_please_leave_me_alonesprings to mind...)?
There is: '__visible'.
-- Luc
Re: [PATCH v3 08/15] arm64: kvm: Split hyp/switch.c to VHE/nVHE
In reply to Marc ZyngierQuoted 38 lines · Marc Zyngier
Quoted 36 lines · kernel test robot
Hi David,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on v5.8-rc2 next-20200624]
[cannot apply to kvmarm/next arm64/for-next/core arm-perf/for-next/perf]
[If your patch is applied to the wrong git tree, kindly drop us a note. And when submitting patch, we suggest to use as documented in
https://git-scm.com/docs/git-format-patch]url:
https://github.com/0day-ci/linux/commits/David-Brazdil/Split-off-nVHE-hyp-code/20200618-203230
base:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git1b5044021070efa3259f3e9548dc35d1eb6aa844
config: arm64-randconfig-r021-20200624 (attached as .config)
compiler: clang version 11.0.0 (https://github.com/llvm/llvm-project8911a35180c6777188fefe0954a2451a2b91deaf)
reproduce (this is a W=1 build):
wget
https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross
-O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install arm64 cross compiling tool for clang build # apt-get install binutils-aarch64-linux-gnu
# save the attached .config to linux build tree COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross
ARCH=arm64If you fix the issue, kindly add following tag as appropriate
Reported-by kernel test robot <[email protected]>All warnings (new ones prefixed by >>):
Quoted 9 lines
Quoted 4 lines
Quoted 2 lines
Quoted 2 lines
arch/arm64/kvm/hyp/nvhe/switch.c:244:28: warning: no previous prototype for function 'hyp_panic' [-Wmissing-prototypes]void
__hyp_text__noreturnhyp_panic(structkvm_cpu_context*host_ctxt)I really wish we could turn these warnings off. They don't add much. Or is there an annotation we could stick on the function (something like
__called_from_asm_please_leave_me_alonesprings to mind...)?
We should define 'asmlinkage' to whatever makes this warning go away.
PATCH v3 9/15 arm64: kvm: Split hyp/debug-sr.c to VHE/nVHE
In reply to David BrazdilThis patch is part of a series which builds KVM's non-VHE hyp code separately from VHE and the rest of the kernel.
debug-sr.c contains KVM's code for context-switching debug registers, with some parts shared between VHE/nVHE. These common routines are moved to debug-sr.h, VHE-specific code is left in debug-sr.c and nVHE-specific code is moved to nvhe/debug-sr.c.
Functions are slightly refactored to move code hidden behind has_vhe() checks to the corresponding .c files.
--- arch/arm64/kernel/image-vars.h | 3 - arch/arm64/kvm/hyp/debug-sr.c | 210 +---------------------------- arch/arm64/kvm/hyp/debug-sr.h | 172 +++++++++++++++++++++++ arch/arm64/kvm/hyp/nvhe/Makefile | 2 +- arch/arm64/kvm/hyp/nvhe/debug-sr.c | 77 +++++++++++ 5 files changed, 256 insertions(+), 208 deletions(-) create mode 100644 arch/arm64/kvm/hyp/debug-sr.h create mode 100644 arch/arm64/kvm/hyp/nvhe/debug-sr.c
arch/arm64/kernel/image-vars.h +0 -3
@@ -61,8 +61,6 @@ __efistub__ctype = _ctype;* memory mappings. */ -__kvm_nvhe___debug_switch_to_guest = __debug_switch_to_guest; -__kvm_nvhe___debug_switch_to_host = __debug_switch_to_host; __kvm_nvhe___fpsimd_restore_state = __fpsimd_restore_state; __kvm_nvhe___fpsimd_save_state = __fpsimd_save_state; __kvm_nvhe___guest_enter = __guest_enter;
@@ -71,7 +69,6 @@ __kvm_nvhe___hyp_panic_string = __hyp_panic_string; __kvm_nvhe___hyp_stub_vectors = __hyp_stub_vectors; __kvm_nvhe___icache_flags = __icache_flags; __kvm_nvhe___kvm_enable_ssbs = __kvm_enable_ssbs; -__kvm_nvhe___kvm_get_mdcr_el2 = __kvm_get_mdcr_el2; __kvm_nvhe___kvm_timer_set_cntvoff = __kvm_timer_set_cntvoff; __kvm_nvhe___sysreg32_restore_state = __sysreg32_restore_state; __kvm_nvhe___sysreg32_save_state = __sysreg32_save_state;
arch/arm64/kvm/hyp/debug-sr.c +6 -204
227 lines not on this page · raw message
arch/arm64/kvm/hyp/debug-sr.h new file +172 -0
172 lines not on this page · raw message
arch/arm64/kvm/hyp/nvhe/Makefile +1 -1
@@ -7,7 +7,7 @@ asflags-y := -D__KVM_NVHE_HYPERVISOR__ ccflags-y := -D__KVM_NVHE_HYPERVISOR__ -fno-stack-protector \ -DDISABLE_BRANCH_PROFILING $(DISABLE_STACKLEAK_PLUGIN) -obj-y := switch.o tlb.o hyp-init.o ../hyp-entry.o +obj-y := debug-sr.o switch.o tlb.o hyp-init.o ../hyp-entry.o obj-y := $(patsubst %.o,%.hyp.o,$(obj-y)) extra-y := $(patsubst %.hyp.o,%.hyp.tmp.o,$(obj-y))
arch/arm64/kvm/hyp/nvhe/debug-sr.c new file +77 -0
77 lines not on this page · raw message
Signature · 6 lines
2.27.0 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v3 10/15 arm64: kvm: Split hyp/sysreg-sr.c to VHE/nVHE
In reply to David BrazdilThis patch is part of a series which builds KVM's non-VHE hyp code separately from VHE and the rest of the kernel.
sysreg-sr.c contains KVM's code for saving/restoring system registers, with some parts shared between VHE/nVHE. These common routines are moved to sysreg-sr.h, VHE-specific code is left in sysreg-sr.c and nVHE-specific code is moved to nvhe/sysreg-sr.c.
--- arch/arm64/include/asm/kvm_hyp.h | 4 + arch/arm64/kernel/image-vars.h | 5 - arch/arm64/kvm/arm.c | 2 +- arch/arm64/kvm/hyp/nvhe/Makefile | 2 +- arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 56 ++++++++ arch/arm64/kvm/hyp/sysreg-sr.c | 215 +--------------------------- arch/arm64/kvm/hyp/sysreg-sr.h | 211 +++++++++++++++++++++++++++ 7 files changed, 279 insertions(+), 216 deletions(-) create mode 100644 arch/arm64/kvm/hyp/nvhe/sysreg-sr.c create mode 100644 arch/arm64/kvm/hyp/sysreg-sr.h
arch/arm64/include/asm/kvm_hyp.h +4 -0
@@ -66,12 +66,16 @@ int __vgic_v3_perform_cpuif_access(struct kvm_vcpu *vcpu);void __timer_enable_traps (struct kvm_vcpu *vcpu);void __timer_disable_traps (struct kvm_vcpu *vcpu); +#ifdef __KVM_NVHE_HYPERVISOR__void __sysreg_save_state_nvhe (struct kvm_cpu_context *ctxt);void __sysreg_restore_state_nvhe (struct kvm_cpu_context *ctxt); +#else void sysreg_save_host_state_vhe (struct kvm_cpu_context *ctxt);void sysreg_restore_host_state_vhe (struct kvm_cpu_context *ctxt);void sysreg_save_guest_state_vhe (struct kvm_cpu_context *ctxt);void sysreg_restore_guest_state_vhe (struct kvm_cpu_context *ctxt); +#endif +void __sysreg32_save_state (struct kvm_vcpu *vcpu);void __sysreg32_restore_state (struct kvm_vcpu *vcpu);
arch/arm64/kernel/image-vars.h +0 -5
@@ -68,12 +68,7 @@ __kvm_nvhe___guest_exit = __guest_exit; __kvm_nvhe___hyp_panic_string = __hyp_panic_string; __kvm_nvhe___hyp_stub_vectors = __hyp_stub_vectors; __kvm_nvhe___icache_flags = __icache_flags; -__kvm_nvhe___kvm_enable_ssbs = __kvm_enable_ssbs; __kvm_nvhe___kvm_timer_set_cntvoff = __kvm_timer_set_cntvoff; -__kvm_nvhe___sysreg32_restore_state = __sysreg32_restore_state; -__kvm_nvhe___sysreg32_save_state = __sysreg32_save_state; -__kvm_nvhe___sysreg_restore_state_nvhe = __sysreg_restore_state_nvhe; -__kvm_nvhe___sysreg_save_state_nvhe = __sysreg_save_state_nvhe; __kvm_nvhe___timer_disable_traps = __timer_disable_traps; __kvm_nvhe___timer_enable_traps = __timer_enable_traps; __kvm_nvhe___vgic_v2_perform_cpuif_access = __vgic_v2_perform_cpuif_access;
arch/arm64/kvm/arm.c +1 -1
@@ -1302,7 +1302,7 @@ static void cpu_init_hyp_mode(void)*/ if (this_cpu_has_cap (ARM64_SSBS ) &&arm64_get_ssbd_state () ==ARM64_SSBD_FORCE_DISABLE ) { -kvm_call_hyp (__kvm_enable_ssbs); +kvm_call_hyp_nvhe (__kvm_enable_ssbs); } }
arch/arm64/kvm/hyp/nvhe/Makefile +1 -1
@@ -7,7 +7,7 @@ asflags-y := -D__KVM_NVHE_HYPERVISOR__ ccflags-y := -D__KVM_NVHE_HYPERVISOR__ -fno-stack-protector \ -DDISABLE_BRANCH_PROFILING $(DISABLE_STACKLEAK_PLUGIN) -obj-y := debug-sr.o switch.o tlb.o hyp-init.o ../hyp-entry.o +obj-y := sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o ../hyp-entry.o obj-y := $(patsubst %.o,%.hyp.o,$(obj-y)) extra-y := $(patsubst %.hyp.o,%.hyp.tmp.o,$(obj-y))
arch/arm64/kvm/hyp/nvhe/sysreg-sr.c new file +56 -0
56 lines not on this page · raw message
arch/arm64/kvm/hyp/sysreg-sr.c +6 -209
247 lines not on this page · raw message
arch/arm64/kvm/hyp/sysreg-sr.h new file +211 -0
211 lines not on this page · raw message
Signature · 6 lines
2.27.0 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v3 11/15 arm64: kvm: Split hyp/timer-sr.c to VHE/nVHE
In reply to David BrazdilThis patch is part of a series which builds KVM's non-VHE hyp code separately from VHE and the rest of the kernel.
timer-sr.c contains a HVC handler for setting CNTVOFF_EL2 and two helper functions for controlling access to physical counter. The former is shared between VHE/nVHE and is kept in timer-sr.c but compiled under both configs. The latter are nVHE-specific and are moved to nvhe/timer-sr.c.
--- arch/arm64/include/asm/kvm_hyp.h | 2 ++ arch/arm64/kernel/image-vars.h | 3 --- arch/arm64/kvm/hyp/nvhe/Makefile | 3 ++- arch/arm64/kvm/hyp/nvhe/timer-sr.c | 43 ++++++++++++++++++++++++++++++ arch/arm64/kvm/hyp/timer-sr.c | 36 ------------------------- 5 files changed, 47 insertions(+), 40 deletions(-) create mode 100644 arch/arm64/kvm/hyp/nvhe/timer-sr.c
arch/arm64/include/asm/kvm_hyp.h +2 -0
@@ -63,8 +63,10 @@ void __vgic_v3_save_aprs(struct vgic_v3_cpu_if *cpu_if);void __vgic_v3_restore_aprs (struct vgic_v3_cpu_if *cpu_if);int __vgic_v3_perform_cpuif_access (struct kvm_vcpu *vcpu); +#ifdef __KVM_NVHE_HYPERVISOR__void __timer_enable_traps (struct kvm_vcpu *vcpu);void __timer_disable_traps (struct kvm_vcpu *vcpu); +#endif #ifdef __KVM_NVHE_HYPERVISOR__void __sysreg_save_state_nvhe (struct kvm_cpu_context *ctxt);
arch/arm64/kernel/image-vars.h +0 -3
@@ -68,9 +68,6 @@ __kvm_nvhe___guest_exit = __guest_exit; __kvm_nvhe___hyp_panic_string = __hyp_panic_string; __kvm_nvhe___hyp_stub_vectors = __hyp_stub_vectors; __kvm_nvhe___icache_flags = __icache_flags; -__kvm_nvhe___kvm_timer_set_cntvoff = __kvm_timer_set_cntvoff; -__kvm_nvhe___timer_disable_traps = __timer_disable_traps; -__kvm_nvhe___timer_enable_traps = __timer_enable_traps; __kvm_nvhe___vgic_v2_perform_cpuif_access = __vgic_v2_perform_cpuif_access; __kvm_nvhe___vgic_v3_activate_traps = __vgic_v3_activate_traps; __kvm_nvhe___vgic_v3_deactivate_traps = __vgic_v3_deactivate_traps;
arch/arm64/kvm/hyp/nvhe/Makefile +2 -1
@@ -7,7 +7,8 @@ asflags-y := -D__KVM_NVHE_HYPERVISOR__ ccflags-y := -D__KVM_NVHE_HYPERVISOR__ -fno-stack-protector \ -DDISABLE_BRANCH_PROFILING $(DISABLE_STACKLEAK_PLUGIN) -obj-y := sysreg-sr.o debug-sr.o switch.o tlb.o hyp-init.o ../hyp-entry.o +obj-y := ../timer-sr.o timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o \ + hyp-init.o ../hyp-entry.o obj-y := $(patsubst %.o,%.hyp.o,$(obj-y)) extra-y := $(patsubst %.hyp.o,%.hyp.tmp.o,$(obj-y))
arch/arm64/kvm/hyp/nvhe/timer-sr.c new file +43 -0
43 lines not on this page · raw message
arch/arm64/kvm/hyp/timer-sr.c +0 -36
@@ -4,45 +4,9 @@* Author: Marc Zyngier <[email protected]> */ -#include <clocksource/arm_arch_timer.h> -#include <linux/compiler.h> -#include <linux/kvm_host.h> -#include <asm/kvm_hyp.h> void __hyp_text __kvm_timer_set_cntvoff(u64 cntvoff) {write_sysreg (cntvoff, cntvoff_el2); } - -/* -* Should only be called on non-VHE systems. -* VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe(). -*/ -void __hyp_text __timer_disable_traps(struct kvm_vcpu *vcpu) -{ -u64 val; - -/* Allow physical timer/counter access for the host */ - val =read_sysreg (cnthctl_el2); - val |=CNTHCTL_EL1PCTEN |CNTHCTL_EL1PCEN ; -write_sysreg (val, cnthctl_el2); -} - -/* -* Should only be called on non-VHE systems. -* VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe(). -*/ -void __hyp_text __timer_enable_traps(struct kvm_vcpu *vcpu) -{ -u64 val; - -/* -* Disallow physical timer access for the guest -* Physical counter access is allowed -*/ - val =read_sysreg (cnthctl_el2); - val &= ~CNTHCTL_EL1PCEN ; - val |=CNTHCTL_EL1PCTEN ; -write_sysreg (val, cnthctl_el2); -}
Signature · 6 lines
2.27.0 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v3 12/15 arm64: kvm: Compile remaining hyp/ files for both VHE/nVHE
In reply to David BrazdilThis patch is part of a series which builds KVM's non-VHE hyp code separately from VHE and the rest of the kernel.
The following files in hyp/ contain only code shared by VHE/nVHE:
vgic-v3-sr.c, aarch32.c, vgic-v2-cpuif-proxy.c, entry.S, fpsimd.S
Compile them under both configurations. Deletions in image-vars.h reflect eliminated dependencies of nVHE code on the rest of the kernel.
--- arch/arm64/kernel/image-vars.h | 19 ------------------- arch/arm64/kvm/hyp/nvhe/Makefile | 5 +++-- 2 files changed, 3 insertions(+), 21 deletions(-)
arch/arm64/kernel/image-vars.h +0 -19
@@ -61,27 +61,9 @@ __efistub__ctype = _ctype;* memory mappings. */ -__kvm_nvhe___fpsimd_restore_state = __fpsimd_restore_state; -__kvm_nvhe___fpsimd_save_state = __fpsimd_save_state; -__kvm_nvhe___guest_enter = __guest_enter; -__kvm_nvhe___guest_exit = __guest_exit; __kvm_nvhe___hyp_panic_string = __hyp_panic_string; __kvm_nvhe___hyp_stub_vectors = __hyp_stub_vectors; __kvm_nvhe___icache_flags = __icache_flags; -__kvm_nvhe___vgic_v2_perform_cpuif_access = __vgic_v2_perform_cpuif_access; -__kvm_nvhe___vgic_v3_activate_traps = __vgic_v3_activate_traps; -__kvm_nvhe___vgic_v3_deactivate_traps = __vgic_v3_deactivate_traps; -__kvm_nvhe___vgic_v3_get_ich_vtr_el2 = __vgic_v3_get_ich_vtr_el2; -__kvm_nvhe___vgic_v3_init_lrs = __vgic_v3_init_lrs; -__kvm_nvhe___vgic_v3_perform_cpuif_access = __vgic_v3_perform_cpuif_access; -__kvm_nvhe___vgic_v3_read_vmcr = __vgic_v3_read_vmcr; -__kvm_nvhe___vgic_v3_restore_aprs = __vgic_v3_restore_aprs; -__kvm_nvhe___vgic_v3_restore_state = __vgic_v3_restore_state; -__kvm_nvhe___vgic_v3_save_aprs = __vgic_v3_save_aprs; -__kvm_nvhe___vgic_v3_save_state = __vgic_v3_save_state; -__kvm_nvhe___vgic_v3_write_vmcr = __vgic_v3_write_vmcr; -__kvm_nvhe_abort_guest_exit_end = abort_guest_exit_end; -__kvm_nvhe_abort_guest_exit_start = abort_guest_exit_start; __kvm_nvhe_arm64_const_caps_ready = arm64_const_caps_ready; __kvm_nvhe_arm64_enable_wa2_handling = arm64_enable_wa2_handling; __kvm_nvhe_arm64_ssbd_callback_required = arm64_ssbd_callback_required;
@@ -94,7 +76,6 @@ __kvm_nvhe_idmap_t0sz = idmap_t0sz; __kvm_nvhe_kimage_voffset = kimage_voffset; __kvm_nvhe_kvm_host_data = kvm_host_data; __kvm_nvhe_kvm_patch_vector_branch = kvm_patch_vector_branch; -__kvm_nvhe_kvm_skip_instr32 = kvm_skip_instr32; __kvm_nvhe_kvm_update_va_mask = kvm_update_va_mask; __kvm_nvhe_kvm_vgic_global_state = kvm_vgic_global_state; __kvm_nvhe_panic = panic;
arch/arm64/kvm/hyp/nvhe/Makefile +3 -2
@@ -7,8 +7,9 @@ asflags-y := -D__KVM_NVHE_HYPERVISOR__ ccflags-y := -D__KVM_NVHE_HYPERVISOR__ -fno-stack-protector \ -DDISABLE_BRANCH_PROFILING $(DISABLE_STACKLEAK_PLUGIN) -obj-y := ../timer-sr.o timer-sr.o sysreg-sr.o debug-sr.o switch.o tlb.o \ - hyp-init.o ../hyp-entry.o +obj-y := ../vgic-v3-sr.o ../timer-sr.o timer-sr.o ../aarch32.o \ + ../vgic-v2-cpuif-proxy.o sysreg-sr.o debug-sr.o ../entry.o switch.o \ + ../fpsimd.o tlb.o hyp-init.o ../hyp-entry.o obj-y := $(patsubst %.o,%.hyp.o,$(obj-y)) extra-y := $(patsubst %.hyp.o,%.hyp.tmp.o,$(obj-y))
Signature · 6 lines
2.27.0 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v3 13/15 arm64: kvm: Add comments around __kvm_nvhe_ symbol aliases
In reply to David BrazdilThis patch is part of a series which builds KVM's non-VHE hyp code separately from VHE and the rest of the kernel.
With all source files split between VHE/nVHE, add comments around the list of symbols where nVHE code still links against kernel proper. Split them into groups and explain how each group is currently used.
Some of these dependencies will be removed in the future.
--- arch/arm64/kernel/image-vars.h | 53 +++++++++++++++++++++------------- 1 file changed, 33 insertions(+), 20 deletions(-)
arch/arm64/kernel/image-vars.h +33 -20
@@ -61,30 +61,43 @@ __efistub__ctype = _ctype;* memory mappings. */ -__kvm_nvhe___hyp_panic_string = __hyp_panic_string; -__kvm_nvhe___hyp_stub_vectors = __hyp_stub_vectors; -__kvm_nvhe___icache_flags = __icache_flags; -__kvm_nvhe_arm64_const_caps_ready = arm64_const_caps_ready; -__kvm_nvhe_arm64_enable_wa2_handling = arm64_enable_wa2_handling; -__kvm_nvhe_arm64_ssbd_callback_required = arm64_ssbd_callback_required; -__kvm_nvhe_cpu_hwcap_keys = cpu_hwcap_keys; -__kvm_nvhe_cpu_hwcaps = cpu_hwcaps; +/* If nVHE code panics, it ERETs into panic() in EL1. */ +__kvm_nvhe___hyp_panic_string = __hyp_panic_string; +__kvm_nvhe_panic = panic; + +/* Values used by the hyp-init vector. */ +__kvm_nvhe___hyp_stub_vectors = __hyp_stub_vectors; +__kvm_nvhe_idmap_t0sz = idmap_t0sz; + +/* Alternative callbacks, referenced in .altinstructions. Executed in EL1. */ +__kvm_nvhe_arm64_enable_wa2_handling = arm64_enable_wa2_handling; +__kvm_nvhe_kvm_patch_vector_branch = kvm_patch_vector_branch; +__kvm_nvhe_kvm_update_va_mask = kvm_update_va_mask; + +/* Values used to convert between memory mappings, read-only after init. */ +__kvm_nvhe_kimage_voffset = kimage_voffset; + +/* Data shared with the kernel. */ +__kvm_nvhe_cpu_hwcaps = cpu_hwcaps; +__kvm_nvhe_cpu_hwcap_keys = cpu_hwcap_keys; +__kvm_nvhe___icache_flags = __icache_flags; +__kvm_nvhe_kvm_vgic_global_state = kvm_vgic_global_state; +__kvm_nvhe_arm64_ssbd_callback_required = arm64_ssbd_callback_required; +__kvm_nvhe_kvm_host_data = kvm_host_data; + +/* Static keys shared with the kernel. */ +__kvm_nvhe_arm64_const_caps_ready = arm64_const_caps_ready;#ifdef CONFIG_ARM64_PSEUDO_NMI -__kvm_nvhe_gic_pmr_sync = gic_pmr_sync; +__kvm_nvhe_gic_pmr_sync = gic_pmr_sync;#endif -__kvm_nvhe_idmap_t0sz = idmap_t0sz; -__kvm_nvhe_kimage_voffset = kimage_voffset; -__kvm_nvhe_kvm_host_data = kvm_host_data; -__kvm_nvhe_kvm_patch_vector_branch = kvm_patch_vector_branch; -__kvm_nvhe_kvm_update_va_mask = kvm_update_va_mask; -__kvm_nvhe_kvm_vgic_global_state = kvm_vgic_global_state; -__kvm_nvhe_panic = panic; +__kvm_nvhe_vgic_v2_cpuif_trap = vgic_v2_cpuif_trap; +__kvm_nvhe_vgic_v3_cpuif_trap = vgic_v3_cpuif_trap; + +/* SVE support, currently unused by nVHE. */ #ifdef CONFIG_ARM64_SVE -__kvm_nvhe_sve_load_state = sve_load_state; -__kvm_nvhe_sve_save_state = sve_save_state; +__kvm_nvhe_sve_save_state = sve_save_state; +__kvm_nvhe_sve_load_state = sve_load_state;#endif -__kvm_nvhe_vgic_v2_cpuif_trap = vgic_v2_cpuif_trap; -__kvm_nvhe_vgic_v3_cpuif_trap = vgic_v3_cpuif_trap;#endif /* CONFIG_KVM */
Signature · 6 lines
2.27.0 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v3 14/15 arm64: kvm: Remove __hyp_text macro, use build rules instead
In reply to David BrazdilWith nVHE code now fully separated from the rest of the kernel, the effects of the __hyp_text macro (which had to be applied on all nVHE code) can be achieved with build rules instead. The macro used to:
(a) move code to .hyp.text ELF section, now done by renaming .text using objcopy, and
(b) notrace and __noscs would negate effects of CC_FLAGS_FTRACE and CC_FLAGS_SCS, respectivelly, now those flags are erased from KBUILD_CFLAGS (same way as in EFI stub).
Note that by removing __hyp_text from code shared with VHE, all VHE code is now compiled into .text and without notrace and __noscs.
Use of '.pushsection .hyp.text' removed from assembly files as this is now also covered by the build rules.
For MAINTAINERS: if needed to re-run, uses of macro were removed with the following command. Formatting was fixed up manually.
find arch/arm64/kvm/hyp -type f -name '*.c' -o -name '*.h' \
-exec sed -i 's/ __hyp_text//g' {} +--- arch/arm64/include/asm/kvm_emulate.h | 2 +- arch/arm64/include/asm/kvm_hyp.h | 2 - arch/arm64/kvm/hyp/aarch32.c | 6 +- arch/arm64/kvm/hyp/debug-sr.h | 18 ++-- arch/arm64/kvm/hyp/entry.S | 1 - arch/arm64/kvm/hyp/fpsimd.S | 1 - arch/arm64/kvm/hyp/hyp-entry.S | 1 - arch/arm64/kvm/hyp/nvhe/Makefile | 8 +- arch/arm64/kvm/hyp/nvhe/debug-sr.c | 10 +- arch/arm64/kvm/hyp/nvhe/switch.c | 18 ++-- arch/arm64/kvm/hyp/nvhe/sysreg-sr.c | 10 +- arch/arm64/kvm/hyp/nvhe/timer-sr.c | 4 +- arch/arm64/kvm/hyp/nvhe/tlb.c | 14 ++- arch/arm64/kvm/hyp/switch.h | 39 ++++--- arch/arm64/kvm/hyp/sysreg-sr.h | 27 ++--- arch/arm64/kvm/hyp/timer-sr.c | 2 +- arch/arm64/kvm/hyp/tlb.c | 6 +- arch/arm64/kvm/hyp/tlb.h | 15 ++- arch/arm64/kvm/hyp/vgic-v2-cpuif-proxy.c | 4 +- arch/arm64/kvm/hyp/vgic-v3-sr.c | 130 ++++++++++------------- 20 files changed, 143 insertions(+), 175 deletions(-)
arch/arm64/include/asm/kvm_emulate.h +1 -1
@@ -516,7 +516,7 @@ static __always_inline void kvm_skip_instr(struct kvm_vcpu *vcpu, bool is_wide_i* Skip an instruction which has been emulated at hyp while most guest sysregs * are live. */ -static __always_inline void __hyp_text __kvm_skip_instr(struct kvm_vcpu *vcpu) +static __always_inline void __kvm_skip_instr(struct kvm_vcpu *vcpu) { *vcpu_pc (vcpu) =read_sysreg_el2 (SYS_ELR ); vcpu->arch.ctxt.gp_regs.regs.pstate =read_sysreg_el2 (SYS_SPSR );
arch/arm64/include/asm/kvm_hyp.h +0 -2
@@ -12,8 +12,6 @@#include <asm/alternative.h> #include <asm/sysreg.h> -#define __hyp_text __section(.hyp.text) notrace __noscs -#define read_sysreg_elx (r,nvh,vh) \ ({ \ u64 reg; \
arch/arm64/kvm/hyp/aarch32.c +3 -3
@@ -44,7 +44,7 @@ static const unsigned short cc_map[16] = {/* * Check if a trapped instruction should have been executed or not. */ -bool __hyp_text kvm_condition_valid32(const struct kvm_vcpu *vcpu) +bool kvm_condition_valid32 (const struct kvm_vcpu *vcpu) {unsigned long cpsr;u32 cpsr_cond;
@@ -93,7 +93,7 @@ bool __hyp_text kvm_condition_valid32(const struct kvm_vcpu *vcpu)* * IT[7:0] -> CPSR[26:25],CPSR[15:10] */ -static void __hyp_text kvm_adjust_itstate(struct kvm_vcpu *vcpu) +static void kvm_adjust_itstate (struct kvm_vcpu *vcpu) {unsigned long itbits, cond;unsigned long cpsr = *vcpu_cpsr (vcpu);
@@ -123,7 +123,7 @@ static void __hyp_text kvm_adjust_itstate(struct kvm_vcpu *vcpu)* kvm_skip_instr - skip a trapped instruction and proceed to the next * @vcpu: The vcpu pointer */ -void __hyp_text kvm_skip_instr32(struct kvm_vcpu *vcpu, bool is_wide_instr) +void kvm_skip_instr32 (struct kvm_vcpu *vcpu,bool is_wide_instr) {u32 pc = *vcpu_pc (vcpu);bool is_thumb;
arch/arm64/kvm/hyp/debug-sr.h +8 -10
@@ -88,9 +88,9 @@default :write_debug (ptr[0 ], reg,0 ); \ } -static inline void __hyp_text -__debug_save_state (struct kvm_vcpu *vcpu,struct kvm_guest_debug_arch *dbg, -struct kvm_cpu_context *ctxt) +static inline void __debug_save_state (struct kvm_vcpu *vcpu, +struct kvm_guest_debug_arch *dbg, +struct kvm_cpu_context *ctxt) {u64 aa64dfr0;int brps, wrps;
@@ -107,9 +107,9 @@ __debug_save_state(struct kvm_vcpu *vcpu, struct kvm_guest_debug_arch *dbg, ctxt->sys_regs[MDCCINT_EL1 ] =read_sysreg (mdccint_el1); } -static inline void __hyp_text -__debug_restore_state (struct kvm_vcpu *vcpu,struct kvm_guest_debug_arch *dbg, -struct kvm_cpu_context *ctxt) +static inline void __debug_restore_state (struct kvm_vcpu *vcpu, +struct kvm_guest_debug_arch *dbg, +struct kvm_cpu_context *ctxt) {u64 aa64dfr0;int brps, wrps;
@@ -127,8 +127,7 @@ __debug_restore_state(struct kvm_vcpu *vcpu, struct kvm_guest_debug_arch *dbg,write_sysreg (ctxt->sys_regs[MDCCINT_EL1 ], mdccint_el1); } -static inline void __hyp_text -__debug_switch_to_guest_common (struct kvm_vcpu *vcpu) +static inline void __debug_switch_to_guest_common (struct kvm_vcpu *vcpu) {struct kvm_cpu_context *host_ctxt;struct kvm_cpu_context *guest_ctxt;
@@ -147,8 +146,7 @@ __debug_switch_to_guest_common(struct kvm_vcpu *vcpu)__debug_restore_state (vcpu, guest_dbg, guest_ctxt); } -static inline void __hyp_text -__debug_switch_to_host_common (struct kvm_vcpu *vcpu) +static inline void __debug_switch_to_host_common (struct kvm_vcpu *vcpu) {struct kvm_cpu_context *host_ctxt;struct kvm_cpu_context *guest_ctxt;
arch/arm64/kvm/hyp/entry.S +0 -1
@@ -21,7 +21,6 @@ #define CPU_SP_EL0_OFFSET (CPU_XREG_OFFSET(30) + 8) .text - .pushsection .hyp.text, "ax" /* * We treat x18 as callee-saved as the host may use it as a platform
arch/arm64/kvm/hyp/fpsimd.S +0 -1
@@ -9,7 +9,6 @@ #include <asm/fpsimdmacros.h> .text - .pushsection .hyp.text, "ax" SYM_FUNC_START(__fpsimd_save_state) fpsimd_save x0, 1
arch/arm64/kvm/hyp/hyp-entry.S +0 -1
@@ -16,7 +16,6 @@ #include <asm/mmu.h> .text - .pushsection .hyp.text, "ax" .macro do_el2_call /*
arch/arm64/kvm/hyp/nvhe/Makefile +7 -1
@@ -22,7 +22,13 @@ $(obj)/%.hyp.o: $(obj)/%.hyp.tmp.o FORCE $(call if_changed,hypcopy) quiet_cmd_hypcopy = HYPCOPY $@ - cmd_hypcopy = $(OBJCOPY) --prefix-symbols=__kvm_nvhe_ $< $@ + cmd_hypcopy = $(OBJCOPY) --prefix-symbols=__kvm_nvhe_ \ + --rename-section=.text=.hyp.text \ + $< $@ + +# Remove ftrace and Shadow Call Stack CFLAGS. +# This is equivalent to the 'notrace' and '__noscs' annotations. +KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS), $(KBUILD_CFLAGS)) # KVM nVHE code is run at a different exception code with a different map, so # compiler instrumentation that inserts callbacks or checks into the code may
arch/arm64/kvm/hyp/nvhe/debug-sr.c +5 -5
@@ -14,7 +14,7 @@#include "../debug-sr.h" -static void __hyp_text __debug_save_spe(u64 *pmscr_el1) +static void __debug_save_spe (u64 *pmscr_el1) {u64 reg;
@@ -46,7 +46,7 @@ static void __hyp_text __debug_save_spe(u64 *pmscr_el1)dsb (nsh); } -static void __hyp_text __debug_restore_spe(u64 pmscr_el1) +static void __debug_restore_spe (u64 pmscr_el1) {if (!pmscr_el1)return ;
@@ -58,20 +58,20 @@ static void __hyp_text __debug_restore_spe(u64 pmscr_el1)write_sysreg_s (pmscr_el1,SYS_PMSCR_EL1 ); } -void __hyp_text __debug_switch_to_guest(struct kvm_vcpu *vcpu) +void __debug_switch_to_guest (struct kvm_vcpu *vcpu) {/* Disable and flush SPE data generation */ __debug_save_spe (&vcpu->arch.host_debug_state.pmscr_el1);__debug_switch_to_guest_common (vcpu); } -void __hyp_text __debug_switch_to_host(struct kvm_vcpu *vcpu) +void __debug_switch_to_host (struct kvm_vcpu *vcpu) {__debug_restore_spe (vcpu->arch.host_debug_state.pmscr_el1);__debug_switch_to_host_common (vcpu); } -u32 __hyp_text __kvm_get_mdcr_el2(void) +u32 __kvm_get_mdcr_el2 (void ) {return read_sysreg (mdcr_el2); }
arch/arm64/kvm/hyp/nvhe/switch.c +9 -9
@@ -26,7 +26,7 @@#include "../switch.h" -static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu) +static void __activate_traps (struct kvm_vcpu *vcpu) {u64 val;
@@ -57,7 +57,7 @@ static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu) } } -static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu) +static void __deactivate_traps (struct kvm_vcpu *vcpu) {u64 mdcr_el2;
@@ -92,13 +92,13 @@ static void __hyp_text __deactivate_traps(struct kvm_vcpu *vcpu)write_sysreg (CPTR_EL2_DEFAULT , cptr_el2); } -static void __hyp_text __deactivate_vm(struct kvm_vcpu *vcpu) +static void __deactivate_vm (struct kvm_vcpu *vcpu) {write_sysreg (0 , vttbr_el2); }/* Save VGICv3 state on non-VHE systems */ -static void __hyp_text __hyp_vgic_save_state(struct kvm_vcpu *vcpu) +static void __hyp_vgic_save_state (struct kvm_vcpu *vcpu) {if (static_branch_unlikely (&kvm_vgic_global_state.gicv3_cpuif)) {__vgic_v3_save_state (&vcpu->arch.vgic_cpu.vgic_v3);
@@ -107,7 +107,7 @@ static void __hyp_text __hyp_vgic_save_state(struct kvm_vcpu *vcpu) }/* Restore VGICv3 state on non_VEH systems */ -static void __hyp_text __hyp_vgic_restore_state(struct kvm_vcpu *vcpu) +static void __hyp_vgic_restore_state (struct kvm_vcpu *vcpu) {if (static_branch_unlikely (&kvm_vgic_global_state.gicv3_cpuif)) {__vgic_v3_activate_traps (&vcpu->arch.vgic_cpu.vgic_v3);
@@ -118,7 +118,7 @@ static void __hyp_text __hyp_vgic_restore_state(struct kvm_vcpu *vcpu)/** * Disable host events, enable guest events */ -static bool __hyp_text __pmu_switch_to_guest(struct kvm_cpu_context *host_ctxt) +static bool __pmu_switch_to_guest (struct kvm_cpu_context *host_ctxt) {struct kvm_host_data *host;struct kvm_pmu_events *pmu;
@@ -138,7 +138,7 @@ static bool __hyp_text __pmu_switch_to_guest(struct kvm_cpu_context *host_ctxt)/** * Disable guest events, enable host events */ -static void __hyp_text __pmu_switch_to_host(struct kvm_cpu_context *host_ctxt) +static void __pmu_switch_to_host (struct kvm_cpu_context *host_ctxt) {struct kvm_host_data *host;struct kvm_pmu_events *pmu;
@@ -154,7 +154,7 @@ static void __hyp_text __pmu_switch_to_host(struct kvm_cpu_context *host_ctxt) }/* Switch to the guest for legacy non-VHE systems */ -int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu) +int __kvm_vcpu_run (struct kvm_vcpu *vcpu) {struct kvm_cpu_context *host_ctxt;struct kvm_cpu_context *guest_ctxt;
@@ -241,7 +241,7 @@ int __hyp_text __kvm_vcpu_run(struct kvm_vcpu *vcpu)return exit_code; } -void __hyp_text __noreturn hyp_panic(struct kvm_cpu_context *host_ctxt) +void __noreturn hyp_panic (struct kvm_cpu_context *host_ctxt) {u64 spsr =read_sysreg_el2 (SYS_SPSR );u64 elr =read_sysreg_el2 (SYS_ELR );
arch/arm64/kvm/hyp/nvhe/sysreg-sr.c +5 -5
@@ -18,7 +18,7 @@* Non-VHE: Both host and guest must save everything. */ -void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt) +void __sysreg_save_state_nvhe (struct kvm_cpu_context *ctxt) {__sysreg_save_el1_state (ctxt);__sysreg_save_common_state (ctxt);
@@ -26,7 +26,7 @@ void __hyp_text __sysreg_save_state_nvhe(struct kvm_cpu_context *ctxt)__sysreg_save_el2_return_state (ctxt); } -void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt) +void __sysreg_restore_state_nvhe (struct kvm_cpu_context *ctxt) {__sysreg_restore_el1_state (ctxt);__sysreg_restore_common_state (ctxt);
@@ -34,17 +34,17 @@ void __hyp_text __sysreg_restore_state_nvhe(struct kvm_cpu_context *ctxt)__sysreg_restore_el2_return_state (ctxt); } -void __hyp_text __sysreg32_save_state(struct kvm_vcpu *vcpu) +void __sysreg32_save_state (struct kvm_vcpu *vcpu) {___sysreg32_save_state (vcpu); } -void __hyp_text __sysreg32_restore_state(struct kvm_vcpu *vcpu) +void __sysreg32_restore_state (struct kvm_vcpu *vcpu) {___sysreg32_restore_state (vcpu); } -void __hyp_text __kvm_enable_ssbs(void) +void __kvm_enable_ssbs (void ) {u64 tmp;
arch/arm64/kvm/hyp/nvhe/timer-sr.c +2 -2
@@ -14,7 +14,7 @@* Should only be called on non-VHE systems. * VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe(). */ -void __hyp_text __timer_disable_traps(struct kvm_vcpu *vcpu) +void __timer_disable_traps (struct kvm_vcpu *vcpu) {u64 val;
@@ -28,7 +28,7 @@ void __hyp_text __timer_disable_traps(struct kvm_vcpu *vcpu)* Should only be called on non-VHE systems. * VHE systems use EL2 timers and configure EL1 timers in kvm_timer_init_vhe(). */ -void __hyp_text __timer_enable_traps(struct kvm_vcpu *vcpu) +void __timer_enable_traps (struct kvm_vcpu *vcpu) {u64 val;
arch/arm64/kvm/hyp/nvhe/tlb.c +6 -8
@@ -12,8 +12,7 @@#include "../tlb.h" -static void __hyp_text __tlb_switch_to_guest(struct kvm *kvm, -struct tlb_inv_context *cxt) +static void __tlb_switch_to_guest (struct kvm *kvm,struct tlb_inv_context *cxt) {if (cpus_have_final_cap (ARM64_WORKAROUND_SPECULATIVE_AT )) {u64 val;
@@ -36,8 +35,7 @@ static void __hyp_text __tlb_switch_to_guest(struct kvm *kvm, asm(ALTERNATIVE("isb", "nop", ARM64_WORKAROUND_SPECULATIVE_AT)); } -static void __hyp_text __tlb_switch_to_host(struct kvm *kvm, -struct tlb_inv_context *cxt) +static void __tlb_switch_to_host (struct kvm *kvm,struct tlb_inv_context *cxt) {write_sysreg (0 , vttbr_el2);
@@ -49,22 +47,22 @@ static void __hyp_text __tlb_switch_to_host(struct kvm *kvm, } } -void __hyp_text __kvm_tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa) +void __kvm_tlb_flush_vmid_ipa (struct kvm *kvm,phys_addr_t ipa) {__tlb_flush_vmid_ipa (kvm, ipa); } -void __hyp_text __kvm_tlb_flush_vmid(struct kvm *kvm) +void __kvm_tlb_flush_vmid (struct kvm *kvm) {__tlb_flush_vmid (kvm); } -void __hyp_text __kvm_tlb_flush_local_vmid(struct kvm_vcpu *vcpu) +void __kvm_tlb_flush_local_vmid (struct kvm_vcpu *vcpu) {__tlb_flush_local_vmid (vcpu); } -void __hyp_text __kvm_flush_vm_context(void) +void __kvm_flush_vm_context (void ) {__tlb_flush_vm_context (); }
arch/arm64/kvm/hyp/switch.h +18 -21
@@ -30,7 +30,7 @@extern const char __hyp_panic_string[];/* Check whether the FP regs were dirtied while in the host-side run loop: */ -static inline bool __hyp_text update_fp_enabled(struct kvm_vcpu *vcpu) +static inline bool update_fp_enabled (struct kvm_vcpu *vcpu) {/* * When the system doesn't support FP/SIMD, we cannot rely on
@@ -48,7 +48,7 @@ static inline bool __hyp_text update_fp_enabled(struct kvm_vcpu *vcpu) }/* Save the 32-bit only FPSIMD system register state */ -static inline void __hyp_text __fpsimd_save_fpexc32(struct kvm_vcpu *vcpu) +static inline void __fpsimd_save_fpexc32 (struct kvm_vcpu *vcpu) {if (!vcpu_el1_is_32bit (vcpu))return ;
@@ -56,7 +56,7 @@ static inline void __hyp_text __fpsimd_save_fpexc32(struct kvm_vcpu *vcpu) vcpu->arch.ctxt.sys_regs[FPEXC32_EL2 ] =read_sysreg (fpexc32_el2); } -static inline void __hyp_text __activate_traps_fpsimd32(struct kvm_vcpu *vcpu) +static inline void __activate_traps_fpsimd32 (struct kvm_vcpu *vcpu) {/* * We are about to set CPTR_EL2.TFP to trap all floating point
@@ -73,7 +73,7 @@ static inline void __hyp_text __activate_traps_fpsimd32(struct kvm_vcpu *vcpu) } } -static inline void __hyp_text __activate_traps_common(struct kvm_vcpu *vcpu) +static inline void __activate_traps_common (struct kvm_vcpu *vcpu) {/* Trap on AArch32 cp15 c15 (impdef sysregs) accesses (EL1 or EL0) */ write_sysreg (1 <<15 , hstr_el2);
@@ -89,13 +89,13 @@ static inline void __hyp_text __activate_traps_common(struct kvm_vcpu *vcpu)write_sysreg (vcpu->arch.mdcr_el2, mdcr_el2); } -static inline void __hyp_text __deactivate_traps_common(void) +static inline void __deactivate_traps_common (void ) {write_sysreg (0 , hstr_el2);write_sysreg (0 , pmuserenr_el0); } -static inline void __hyp_text ___activate_traps(struct kvm_vcpu *vcpu) +static inline void ___activate_traps (struct kvm_vcpu *vcpu) {u64 hcr = vcpu->arch.hcr_el2;
@@ -108,7 +108,7 @@ static inline void __hyp_text ___activate_traps(struct kvm_vcpu *vcpu)write_sysreg_s (vcpu->arch.vsesr_el2,SYS_VSESR_EL2 ); } -static inline void __hyp_text ___deactivate_traps(struct kvm_vcpu *vcpu) +static inline void ___deactivate_traps (struct kvm_vcpu *vcpu) {/* * If we pended a virtual abort, preserve it until it gets
@@ -122,12 +122,12 @@ static inline void __hyp_text ___deactivate_traps(struct kvm_vcpu *vcpu) } } -static inline void __hyp_text __activate_vm(struct kvm *kvm) +static inline void __activate_vm (struct kvm *kvm) {__load_guest_stage2 (kvm); } -static inline bool __hyp_text __translate_far_to_hpfar(u64 far, u64 *hpfar) +static inline bool __translate_far_to_hpfar (u64 far,u64 *hpfar) {u64 par, tmp;
@@ -156,7 +156,7 @@ static inline bool __hyp_text __translate_far_to_hpfar(u64 far, u64 *hpfar)return true; } -static inline bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu) +static inline bool __populate_fault_info (struct kvm_vcpu *vcpu) {u8 ec;u64 esr;
@@ -196,7 +196,7 @@ static inline bool __hyp_text __populate_fault_info(struct kvm_vcpu *vcpu) }/* Check for an FPSIMD/SVE trap and handle as appropriate */ -static inline bool __hyp_text __hyp_handle_fpsimd(struct kvm_vcpu *vcpu) +static inline bool __hyp_handle_fpsimd (struct kvm_vcpu *vcpu) {bool vhe, sve_guest, sve_host;u8 hsr_ec;
@@ -278,7 +278,7 @@ static inline bool __hyp_text __hyp_handle_fpsimd(struct kvm_vcpu *vcpu)return true; } -static inline bool __hyp_text handle_tx2_tvm(struct kvm_vcpu *vcpu) +static inline bool handle_tx2_tvm (struct kvm_vcpu *vcpu) {u32 sysreg =esr_sys64_to_sysreg (kvm_vcpu_get_hsr (vcpu));int rt =kvm_vcpu_sys_get_rt (vcpu);
@@ -333,7 +333,7 @@ static inline bool __hyp_text handle_tx2_tvm(struct kvm_vcpu *vcpu)return true; } -static inline bool __hyp_text esr_is_ptrauth_trap(u32 esr) +static inline bool esr_is_ptrauth_trap (u32 esr) {u32 ec =ESR_ELx_EC (esr);
@@ -366,7 +366,7 @@ static inline bool __hyp_text esr_is_ptrauth_trap(u32 esr) regs[key ## KEYHI_EL1] = read_sysreg_s(SYS_ ## key ## KEYHI_EL1); \ }) -static inline bool __hyp_text __hyp_handle_ptrauth(struct kvm_vcpu *vcpu) +static inline bool __hyp_handle_ptrauth (struct kvm_vcpu *vcpu) {struct kvm_cpu_context *ctxt;u64 val;
@@ -396,8 +396,7 @@ static inline bool __hyp_text __hyp_handle_ptrauth(struct kvm_vcpu *vcpu)* the guest, false when we should restore the host state and return to the * main run loop. */ -static inline bool __hyp_text -fixup_guest_exit (struct kvm_vcpu *vcpu,u64 *exit_code) +static inline bool fixup_guest_exit (struct kvm_vcpu *vcpu,u64 *exit_code) {if (ARM_EXCEPTION_CODE (*exit_code) !=ARM_EXCEPTION_IRQ ) vcpu->arch.fault.esr_el2 =read_sysreg_el2 (SYS_ESR );
@@ -469,7 +468,7 @@ fixup_guest_exit(struct kvm_vcpu *vcpu, u64 *exit_code)return false; } -static inline bool __hyp_text __needs_ssbd_off(struct kvm_vcpu *vcpu) +static inline bool __needs_ssbd_off (struct kvm_vcpu *vcpu) {if (!cpus_have_final_cap (ARM64_SSBD ))return false;
@@ -477,8 +476,7 @@ static inline bool __hyp_text __needs_ssbd_off(struct kvm_vcpu *vcpu)return !(vcpu->arch.workaround_flags &VCPU_WORKAROUND_2_FLAG ); } -static inline void __hyp_text -__set_guest_arch_workaround_state (struct kvm_vcpu *vcpu) +static inline void __set_guest_arch_workaround_state (struct kvm_vcpu *vcpu) {#ifdef CONFIG_ARM64_SSBD /*
@@ -491,8 +489,7 @@ __set_guest_arch_workaround_state(struct kvm_vcpu *vcpu)#endif } -static inline void __hyp_text -__set_host_arch_workaround_state (struct kvm_vcpu *vcpu) +static inline void __set_host_arch_workaround_state (struct kvm_vcpu *vcpu) {#ifdef CONFIG_ARM64_SSBD /*
arch/arm64/kvm/hyp/sysreg-sr.h +10 -17
@@ -15,21 +15,18 @@#include <asm/kvm_emulate.h> #include <asm/kvm_hyp.h> -static inline void __hyp_text -__sysreg_save_common_state (struct kvm_cpu_context *ctxt) +static inline void __sysreg_save_common_state (struct kvm_cpu_context *ctxt) { ctxt->sys_regs[MDSCR_EL1 ] =read_sysreg (mdscr_el1); } -static inline void __hyp_text -__sysreg_save_user_state (struct kvm_cpu_context *ctxt) +static inline void __sysreg_save_user_state (struct kvm_cpu_context *ctxt) { ctxt->sys_regs[TPIDR_EL0 ] =read_sysreg (tpidr_el0); ctxt->sys_regs[TPIDRRO_EL0 ] =read_sysreg (tpidrro_el0); } -static inline void __hyp_text -__sysreg_save_el1_state (struct kvm_cpu_context *ctxt) +static inline void __sysreg_save_el1_state (struct kvm_cpu_context *ctxt) { ctxt->sys_regs[CSSELR_EL1 ] =read_sysreg (csselr_el1); ctxt->sys_regs[SCTLR_EL1 ] =read_sysreg_el1 (SYS_SCTLR );
@@ -54,8 +51,7 @@ __sysreg_save_el1_state(struct kvm_cpu_context *ctxt) ctxt->gp_regs.spsr[KVM_SPSR_EL1 ]=read_sysreg_el1 (SYS_SPSR ); } -static inline void __hyp_text -__sysreg_save_el2_return_state (struct kvm_cpu_context *ctxt) +static inline void __sysreg_save_el2_return_state (struct kvm_cpu_context *ctxt) { ctxt->gp_regs.regs.pc =read_sysreg_el2 (SYS_ELR ); ctxt->gp_regs.regs.pstate =read_sysreg_el2 (SYS_SPSR );
@@ -64,21 +60,18 @@ __sysreg_save_el2_return_state(struct kvm_cpu_context *ctxt) ctxt->sys_regs[DISR_EL1 ] =read_sysreg_s (SYS_VDISR_EL2 ); } -static inline void __hyp_text -__sysreg_restore_common_state (struct kvm_cpu_context *ctxt) +static inline void __sysreg_restore_common_state (struct kvm_cpu_context *ctxt) {write_sysreg (ctxt->sys_regs[MDSCR_EL1 ], mdscr_el1); } -static inline void __hyp_text -__sysreg_restore_user_state (struct kvm_cpu_context *ctxt) +static inline void __sysreg_restore_user_state (struct kvm_cpu_context *ctxt) {write_sysreg (ctxt->sys_regs[TPIDR_EL0 ], tpidr_el0);write_sysreg (ctxt->sys_regs[TPIDRRO_EL0 ], tpidrro_el0); } -static inline void __hyp_text -__sysreg_restore_el1_state (struct kvm_cpu_context *ctxt) +static inline void __sysreg_restore_el1_state (struct kvm_cpu_context *ctxt) {write_sysreg (ctxt->sys_regs[MPIDR_EL1 ], vmpidr_el2);write_sysreg (ctxt->sys_regs[CSSELR_EL1 ], csselr_el1);
@@ -137,7 +130,7 @@ __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt)write_sysreg_el1 (ctxt->gp_regs.spsr[KVM_SPSR_EL1 ],SYS_SPSR ); } -static inline void __hyp_text +static inline void __sysreg_restore_el2_return_state (struct kvm_cpu_context *ctxt) {u64 pstate = ctxt->gp_regs.regs.pstate;
@@ -164,7 +157,7 @@ __sysreg_restore_el2_return_state(struct kvm_cpu_context *ctxt)write_sysreg_s (ctxt->sys_regs[DISR_EL1 ],SYS_VDISR_EL2 ); } -static inline void __hyp_text ___sysreg32_save_state(struct kvm_vcpu *vcpu) +static inline void ___sysreg32_save_state (struct kvm_vcpu *vcpu) {u64 *spsr, *sysreg;
@@ -186,7 +179,7 @@ static inline void __hyp_text ___sysreg32_save_state(struct kvm_vcpu *vcpu) sysreg[DBGVCR32_EL2 ] =read_sysreg (dbgvcr32_el2); } -static inline void __hyp_text ___sysreg32_restore_state(struct kvm_vcpu *vcpu) +static inline void ___sysreg32_restore_state (struct kvm_vcpu *vcpu) {u64 *spsr, *sysreg;
arch/arm64/kvm/hyp/timer-sr.c +1 -1
@@ -6,7 +6,7 @@#include <asm/kvm_hyp.h> -void __hyp_text __kvm_timer_set_cntvoff(u64 cntvoff) +void __kvm_timer_set_cntvoff (u64 cntvoff) {write_sysreg (cntvoff, cntvoff_el2); }
arch/arm64/kvm/hyp/tlb.c +2 -4
@@ -12,8 +12,7 @@#include "tlb.h" -static void __hyp_text __tlb_switch_to_guest(struct kvm *kvm, -struct tlb_inv_context *cxt) +static void __tlb_switch_to_guest (struct kvm *kvm,struct tlb_inv_context *cxt) {u64 val;
@@ -56,8 +55,7 @@ static void __hyp_text __tlb_switch_to_guest(struct kvm *kvm,isb (); } -static void __hyp_text __tlb_switch_to_host(struct kvm *kvm, -struct tlb_inv_context *cxt) +static void __tlb_switch_to_host (struct kvm *kvm,struct tlb_inv_context *cxt) {/* * We're done with the TLB operation, let's restore the host's
arch/arm64/kvm/hyp/tlb.h +6 -9
@@ -19,13 +19,10 @@ struct tlb_inv_context {u64 sctlr; }; -static void __hyp_text __tlb_switch_to_guest(struct kvm *kvm, -struct tlb_inv_context *cxt); -static void __hyp_text __tlb_switch_to_host(struct kvm *kvm, -struct tlb_inv_context *cxt); +static void __tlb_switch_to_guest (struct kvm *kvm,struct tlb_inv_context *cxt); +static void __tlb_switch_to_host (struct kvm *kvm,struct tlb_inv_context *cxt); -static inline void __hyp_text -__tlb_flush_vmid_ipa (struct kvm *kvm,phys_addr_t ipa) +static inline void __tlb_flush_vmid_ipa (struct kvm *kvm,phys_addr_t ipa) {struct tlb_inv_context cxt;
@@ -79,7 +76,7 @@ __tlb_flush_vmid_ipa(struct kvm *kvm, phys_addr_t ipa)__tlb_switch_to_host (kvm, &cxt); } -static inline void __hyp_text __tlb_flush_vmid(struct kvm *kvm) +static inline void __tlb_flush_vmid (struct kvm *kvm) {struct tlb_inv_context cxt;
@@ -96,7 +93,7 @@ static inline void __hyp_text __tlb_flush_vmid(struct kvm *kvm)__tlb_switch_to_host (kvm, &cxt); } -static inline void __hyp_text __tlb_flush_local_vmid(struct kvm_vcpu *vcpu) +static inline void __tlb_flush_local_vmid (struct kvm_vcpu *vcpu) {struct kvm *kvm =kern_hyp_va (kern_hyp_va (vcpu)->kvm);struct tlb_inv_context cxt;
@@ -111,7 +108,7 @@ static inline void __hyp_text __tlb_flush_local_vmid(struct kvm_vcpu *vcpu)__tlb_switch_to_host (kvm, &cxt); } -static inline void __hyp_text __tlb_flush_vm_context(void) +static inline void __tlb_flush_vm_context (void ) {dsb (ishst);__tlbi (alle1is);
arch/arm64/kvm/hyp/vgic-v2-cpuif-proxy.c +2 -2
@@ -13,7 +13,7 @@#include <asm/kvm_hyp.h> #include <asm/kvm_mmu.h> -static bool __hyp_text __is_be(struct kvm_vcpu *vcpu) +static bool __is_be (struct kvm_vcpu *vcpu) {if (vcpu_mode_is_32bit (vcpu))return !!(read_sysreg_el2 (SYS_SPSR ) &PSR_AA32_E_BIT );
@@ -32,7 +32,7 @@ static bool __hyp_text __is_be(struct kvm_vcpu *vcpu)* 0: Not a GICV access * -1: Illegal GICV access successfully performed */ -int __hyp_text __vgic_v2_perform_cpuif_access(struct kvm_vcpu *vcpu) +int __vgic_v2_perform_cpuif_access (struct kvm_vcpu *vcpu) {struct kvm *kvm =kern_hyp_va (vcpu->kvm);struct vgic_dist *vgic = &kvm->arch.vgic;
arch/arm64/kvm/hyp/vgic-v3-sr.c +58 -72
434 lines not on this page · raw message
Signature · 6 lines
2.27.0 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v3 15/15 arm64: kvm: Lift instrumentation restrictions on VHE
In reply to David BrazdilWith VHE and nVHE executable code completely separated, remove build config that disabled GCOV/KASAN/UBSAN/KCOV instrumentation for VHE as these now execute under the same memory mappings as the rest of the kernel.
No violations are currently being reported by either KASAN or UBSAN.
--- arch/arm64/kvm/hyp/Makefile | 8 -------- 1 file changed, 8 deletions(-)
arch/arm64/kvm/hyp/Makefile +0 -8
@@ -11,11 +11,3 @@ obj-$(CONFIG_KVM_INDIRECT_VECTORS) += smccc_wa.o hyp-y := vgic-v3-sr.o timer-sr.o aarch32.o vgic-v2-cpuif-proxy.o sysreg-sr.o \ debug-sr.o entry.o switch.o fpsimd.o tlb.o hyp-entry.o - -# KVM code is run at a different exception code with a different map, so -# compiler instrumentation that inserts callbacks or checks into the code may -# cause crashes. Just disable it. -GCOV_PROFILE := n -KASAN_SANITIZE := n -UBSAN_SANITIZE := n -KCOV_INSTRUMENT := n
Signature · 6 lines
2.27.0 _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm