KVM: arm64: UBSAN at EL2
Subsystems (6)
v2 cover KVM: arm64: UBSAN at EL2
Many of the sanitizers the kernel supports are disabled when running in EL2 with nvhe/hvhe/proctected modes, some of those are easier (and makes more sense) to integrate than others.
Last year, kCFI support was added in [1]
This patchset adds support for UBSAN in EL2.
UBSAN can run in 2 modes:
1) “Normal” (CONFIG_UBSAN_TRAP=n): In this mode the compiler will do the UBSAN checks and insert some function calls in case of failures, it can provide more information(ex: what is the value of the out of bound) about the failures through those function arguments, and those functions(implemented in lib/ubsan.c) will print a report with such errors.
2) Trap (CONFIG_UBSAN_TRAP=y): This is a minimal mode, where similarly, the compiler will do the checks, but instead of doing function calls, it would do a “brk #imm” (for ARM64) with a unique code with the failure type, but without any extra information (ex: only print the out-bound line but not the index)
For nvhe/hvhe/proctected modes, #2 would be suitable, as there is no way to print reports from EL2, so similarly to kCFI(even with permissive) it would cause the hypervisor to panic.
But that means that for EL2 we need to compile the code with the same options as used by “CONFIG_UBSAN_TRAP” independently from the kernel config.
This patch series adds a new KCONFIG for ARM64 to choose to enable UBSAN separately for the modes mentioned.
The same logic decoding the kernel UBSAN is reused, so the messages from the hypervisor will look similar as:
[ 29.215332] kvm [190]: nVHE hyp UBSAN: array index out of bounds at: [<ffff8000811f2344>] __kvm_nvhe_handle___pkvm_init_vm+0xa8/0xac!
In this patch set, the same UBSAN options(for check types) are used for both EL1/EL2, although a case can be made to have separate options (leading to totally separate CFLAGS) if we want EL2 to be compiled with stricter checks for something as protected mode.
However, re-using the current flags, makes code re-use easier for report_ubsan_failure() and Makefile.ubsan
[1] https://lore.kernel.org/all/[email protected]/
Changes from v1:
- https://lore.kernel.org/all/[email protected]/
- Collected Kees Acked-By
- Rename CFLAGS flag to CFLAGS_UBSAN_TRAP
- Small comment fix
Mostafa Saleh (4):
arm64: Introduce esr_is_ubsan_brk()
ubsan: Remove regs from report_ubsan_failure()
KVM: arm64: Introduce CONFIG_UBSAN_KVM_EL2
KVM: arm64: Handle UBSAN faults
arch/arm64/include/asm/esr.h | 5 +++++ arch/arm64/kernel/traps.c | 4 ++-- arch/arm64/kvm/handle_exit.c | 6 ++++++ arch/arm64/kvm/hyp/nvhe/Makefile | 6 ++++++ arch/x86/kernel/traps.c | 2 +- include/linux/ubsan.h | 6 +++--- lib/Kconfig.ubsan | 9 +++++++++ lib/ubsan.c | 8 +++++--- scripts/Makefile.ubsan | 5 ++++- 9 files changed, 41 insertions(+), 10 deletions(-)
Signature · 1 line
2.49.0.967.g6a0df3ecc3-goog
PATCH v2 1/4 arm64: Introduce esr_is_ubsan_brk()
Soon, KVM is going to use this logic for hypervisor panics, so add it in a wrapper that can be used by the hypervisor exit handler to decode hyp panics.
--- arch/arm64/include/asm/esr.h | 5 +++++ arch/arm64/kernel/traps.c | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-)
arch/arm64/include/asm/esr.h +5 -0
@@ -440,6 +440,11 @@ static inline bool esr_is_cfi_brk(unsigned long esr) (esr_brk_comment (esr) & ~CFI_BRK_IMM_MASK ) ==CFI_BRK_IMM_BASE ; } +static inline bool esr_is_ubsan_brk (unsigned long esr) +{ +return (esr_brk_comment (esr) & ~UBSAN_BRK_MASK ) ==UBSAN_BRK_IMM ; +} +static inline bool esr_fsc_is_translation_fault (unsigned long esr) { esr = esr & ESR_ELx_FSC;
arch/arm64/kernel/traps.c +1 -1
@@ -1145,7 +1145,7 @@ int __init early_brk64(unsigned long addr, unsigned long esr,return kasan_handler (regs, esr) !=DBG_HOOK_HANDLED ;#endif #ifdef CONFIG_UBSAN_TRAP -if ((esr_brk_comment (esr) & ~UBSAN_BRK_MASK ) ==UBSAN_BRK_IMM ) +if (esr_is_ubsan_brk (esr))return ubsan_handler (regs, esr) !=DBG_HOOK_HANDLED ;#endif return bug_handler (regs, esr) !=DBG_HOOK_HANDLED ;
Signature · 1 line
2.49.0.967.g6a0df3ecc3-goog
Re: [PATCH v2 1/4] arm64: Introduce esr_is_ubsan_brk()
Quoted 5 lines · Mostafa Saleh
Soon, KVM is going to use this logic for hypervisor panics, so add it in a wrapper that can be used by the hypervisor exit handler to decode hyp panics.
Signed-off-by Mostafa Saleh <[email protected]>
Mechanical change; looks good.
Signature · 1 line
Kees Cook
PATCH v2 2/4 ubsan: Remove regs from report_ubsan_failure()
In reply to Mostafa Salehreport_ubsan_failure() doesn't use argument regs, and soon it will be called from the hypervisor context were regs are not available. So, remove the unused argument.
--- arch/arm64/kernel/traps.c | 2 +- arch/x86/kernel/traps.c | 2 +- include/linux/ubsan.h | 4 ++-- lib/ubsan.c | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-)
arch/arm64/kernel/traps.c +1 -1
@@ -1118,7 +1118,7 @@ static struct break_hook kasan_break_hook = {#ifdef CONFIG_UBSAN_TRAP static int ubsan_handler (struct pt_regs *regs,unsigned long esr) { -die (report_ubsan_failure (regs, esr &UBSAN_BRK_MASK ), regs, esr); +die (report_ubsan_failure (esr &UBSAN_BRK_MASK ), regs, esr);return DBG_HOOK_HANDLED ; }
arch/x86/kernel/traps.c +1 -1
@@ -351,7 +351,7 @@ static noinstr bool handle_bug(struct pt_regs *regs)case BUG_UD1_UBSAN :if (IS_ENABLED (CONFIG_UBSAN_TRAP )) {pr_crit ("%s at %pS\n" , -report_ubsan_failure (regs, ud_imm), +report_ubsan_failure (ud_imm), (void *)regs->ip); }break ;
include/linux/ubsan.h +2 -2
@@ -3,9 +3,9 @@#define _LINUX_UBSAN_H#ifdef CONFIG_UBSAN_TRAP -const char *report_ubsan_failure (struct pt_regs *regs,u32 check_type); +const char *report_ubsan_failure (u32 check_type);#else -static inline const char *report_ubsan_failure (struct pt_regs *regs,u32 check_type) +static inline const char *report_ubsan_failure (u32 check_type) {return NULL ; }
lib/ubsan.c +1 -1
@@ -25,7 +25,7 @@* The mappings of struct SanitizerKind (the -fsanitize=xxx args) to * enum SanitizerHandler (the traps) in Clang is in clang/lib/CodeGen/. */ -const char *report_ubsan_failure (struct pt_regs *regs,u32 check_type) +const char *report_ubsan_failure (u32 check_type) {switch (check_type) {#ifdef CONFIG_UBSAN_BOUNDS
Signature · 1 line
2.49.0.967.g6a0df3ecc3-goog
PATCH v2 3/4 KVM: arm64: Introduce CONFIG_UBSAN_KVM_EL2
In reply to Mostafa SalehAdd a new Kconfig CONFIG_UBSAN_KVM_EL2 for KVM which enables UBSAN for EL2 code (in protected/nvhe/hvhe) modes. This will re-use the same checks enabled for the kernel for the hypervisor. The only difference is that for EL2 it always emits a "brk" instead of implementing hooks as the hypervisor can't print reports.
The KVM code will re-use the same code for the kernel "report_ubsan_failure()" so #ifdefs are changed to also have this code for CONFIG_UBSAN_KVM_EL2
--- arch/arm64/kvm/hyp/nvhe/Makefile | 6 ++++++ include/linux/ubsan.h | 2 +- lib/Kconfig.ubsan | 9 +++++++++ lib/ubsan.c | 6 ++++-- scripts/Makefile.ubsan | 5 ++++- 5 files changed, 24 insertions(+), 4 deletions(-)
arch/arm64/kvm/hyp/nvhe/Makefile +6 -0
@@ -99,3 +99,9 @@ KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS), $(KBUILD_CFLAG # causes a build failure. Remove profile optimization flags. KBUILD_CFLAGS := $(filter-out -fprofile-sample-use=% -fprofile-use=%, $(KBUILD_CFLAGS)) KBUILD_CFLAGS += -fno-asynchronous-unwind-tables -fno-unwind-tables + +ifeq ($(CONFIG_UBSAN_KVM_EL2),y) +UBSAN_SANITIZE := y +# Always use brk and not hooks +ccflags-y += $(CFLAGS_UBSAN_TRAP) +endif
include/linux/ubsan.h +1 -1
@@ -2,7 +2,7 @@#ifndef _LINUX_UBSAN_H#define _LINUX_UBSAN_H -#ifdef CONFIG_UBSAN_TRAP +#if defined(CONFIG_UBSAN_TRAP ) || defined(CONFIG_UBSAN_KVM_EL2 )const char *report_ubsan_failure (u32 check_type);#else static inline const char *report_ubsan_failure (u32 check_type)
lib/Kconfig.ubsan +9 -0
@@ -165,4 +165,13 @@ config TEST_UBSAN This is a test module for UBSAN. It triggers various undefined behavior, and detect it. +config UBSAN_KVM_EL2 + bool "UBSAN for KVM code at EL2" + depends on ARM64 + help + Enable UBSAN when running on ARM64 with KVM in a split mode + (nvhe/hvhe/protected) for the hypervisor code running in EL2. + In this mode, any UBSAN violation in EL2 would panic the kernel + and information similar to UBSAN_TRAP would be printed. + endif # if UBSAN
lib/ubsan.c +4 -2
@@ -19,7 +19,7 @@#include "ubsan.h" -#ifdef CONFIG_UBSAN_TRAP +#if defined(CONFIG_UBSAN_TRAP ) || defined(CONFIG_UBSAN_KVM_EL2 )/* * Only include matches for UBSAN checks that are actually compiled in. * The mappings of struct SanitizerKind (the -fsanitize=xxx args) to
@@ -97,7 +97,9 @@ const char *report_ubsan_failure(u32 check_type) } } -#else +#endif + +#ifndef CONFIG_UBSAN_TRAP static const char *const type_check_kinds[] = {"load of" , "store to",
scripts/Makefile.ubsan +4 -1
@@ -1,5 +1,8 @@ # SPDX-License-Identifier: GPL-2.0 +# Shared with KVM/arm64. +export CFLAGS_UBSAN_TRAP := $(call cc-option,-fsanitize-trap=undefined,-fsanitize-undefined-trap-on-error) + # Enable available and selected UBSAN features. ubsan-cflags-$(CONFIG_UBSAN_ALIGNMENT) += -fsanitize=alignment ubsan-cflags-$(CONFIG_UBSAN_BOUNDS_STRICT) += -fsanitize=bounds-strict
@@ -10,7 +13,7 @@ ubsan-cflags-$(CONFIG_UBSAN_DIV_ZERO) += -fsanitize=integer-divide-by-zero ubsan-cflags-$(CONFIG_UBSAN_UNREACHABLE) += -fsanitize=unreachable ubsan-cflags-$(CONFIG_UBSAN_BOOL) += -fsanitize=bool ubsan-cflags-$(CONFIG_UBSAN_ENUM) += -fsanitize=enum -ubsan-cflags-$(CONFIG_UBSAN_TRAP) += $(call cc-option,-fsanitize-trap=undefined,-fsanitize-undefined-trap-on-error) +ubsan-cflags-$(CONFIG_UBSAN_TRAP) += $(CFLAGS_UBSAN_TRAP) export CFLAGS_UBSAN := $(ubsan-cflags-y)
Signature · 1 line
2.49.0.967.g6a0df3ecc3-goog
Re: [PATCH v2 3/4] KVM: arm64: Introduce CONFIG_UBSAN_KVM_EL2
Quoted 7 lines · Mostafa Saleh
Add a new
KconfigCONFIG_UBSAN_KVM_EL2for KVM which enables UBSAN for EL2 code (in protected/nvhe/hvhe) modes. This will re-use the same checks enabled for the kernel for the hypervisor. The only difference is that for EL2 it always emits a "brk" instead of implementing hooks as the hypervisor can't print reports.
Quoted 5 lines
The KVM code will re-use the same code for the kernel "
report_ubsan_failure()" so #ifdefs are changed to also have this code forCONFIG_UBSAN_KVM_EL2Signed-off-by Mostafa Saleh <[email protected]>
Thanks for the rename, this looks good!
Signature · 1 line
Kees Cook
PATCH v2 4/4 KVM: arm64: Handle UBSAN faults
In reply to Mostafa SalehAs now UBSAN can be enabled, handle brk64 exits from UBSAN. Re-use the decoding code from the kernel, and panic with UBSAN message.
--- arch/arm64/kvm/handle_exit.c | 6 ++++++ 1 file changed, 6 insertions(+)
arch/arm64/kvm/handle_exit.c +6 -0
@@ -10,6 +10,7 @@#include <linux/kvm.h> #include <linux/kvm_host.h> +#include <linux/ubsan.h> #include <asm/esr.h> #include <asm/exception.h>
@@ -474,6 +475,11 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr,print_nvhe_hyp_panic ("BUG" , panic_addr); }else if (IS_ENABLED (CONFIG_CFI_CLANG ) &&esr_is_cfi_brk (esr)) {kvm_nvhe_report_cfi_failure (panic_addr); + }else if (IS_ENABLED (CONFIG_UBSAN_KVM_EL2 ) && +ESR_ELx_EC (esr) == ESR_ELx_EC_BRK64 && +esr_is_ubsan_brk (esr)) { +print_nvhe_hyp_panic (report_ubsan_failure (esr &UBSAN_BRK_MASK ), + panic_addr); }else {print_nvhe_hyp_panic ("panic" , panic_addr); }
Signature · 1 line
2.49.0.967.g6a0df3ecc3-goog
Re: [PATCH v2 4/4] KVM: arm64: Handle UBSAN faults
Quoted 5 lines · Mostafa Saleh
As now UBSAN can be enabled, handle brk64 exits from UBSAN. Re-use the decoding code from the kernel, and panic with UBSAN message.
Signed-off-by Mostafa Saleh <[email protected]>
Looks correct to me.
Signature · 1 line
Kees Cook
Kees Cook
maintainerQuoted 6 lines · Mostafa Saleh
Many of the sanitizers the kernel supports are disabled when running in EL2 with nvhe/hvhe/proctected modes, some of those are easier (and makes more sense) to integrate than others.
Last year, kCFI support was added in [1]This patchset adds support for UBSAN in EL2.
This touches both UBSAN and arm64 -- I'm happy to land this via the hardening tree, but I expect the arm64 folks would rather take it via their tree. What would people like to have happen?
-Kees
Signature · 1 line
Kees Cook
Marc Zyngier
maintainerQuoted 7 lines · Kees Cook
Quoted 5 lines · Mostafa Saleh
Many of the sanitizers the kernel supports are disabled when running in EL2 with nvhe/hvhe/proctected modes, some of those are easier (and makes more sense) to integrate than others.
Last year, kCFI support was added in [1]
Quoted 5 lines
Quoted 1 line
This patchset adds support for UBSAN in EL2.
This touches both UBSAN and arm64 -- I'm happy to land this via the hardening tree, but I expect the arm64 folks would rather take it via their tree. What would people like to have happen?
I don't mind either way, but in any case I'd like a stable branch with that code so that I can merge it if any conflict occurs in -next.
Alternatively, I can take it via the kvmarm tree, and publish a stable branch for anyone to pick and resolve conflicts ahead of the merge window.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Marc Zyngier
maintainerQuoted 7 lines · Kees Cook
Quoted 5 lines · Mostafa Saleh
Many of the sanitizers the kernel supports are disabled when running in EL2 with nvhe/hvhe/proctected modes, some of those are easier (and makes more sense) to integrate than others.
Last year, kCFI support was added in [1]
Quoted 5 lines
Quoted 1 line
This patchset adds support for UBSAN in EL2.
This touches both UBSAN and arm64 -- I'm happy to land this via the hardening tree, but I expect the arm64 folks would rather take it via their tree. What would people like to have happen?
FWIW, I have now taken this in kvmarm/next. A stable branch is available at [1] for anyone to pull and resolve potential conflicts.
Thanks,
M.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git/log/?h=kvm-arm64/ubsan-el2
Signature · 1 line
Without deviation from the norm, progress is not possible.
Kees Cook
maintainerQuoted 9 lines · Marc Zyngier
Quoted 7 lines · Kees Cook
Quoted 5 lines · Mostafa Saleh
Many of the sanitizers the kernel supports are disabled when running in EL2 with nvhe/hvhe/proctected modes, some of those are easier (and makes more sense) to integrate than others.
Last year, kCFI support was added in [1]
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
This patchset adds support for UBSAN in EL2.
This touches both UBSAN and arm64 -- I'm happy to land this via the hardening tree, but I expect the arm64 folks would rather take it via their tree. What would people like to have happen?
FWIW, I have now taken this in kvmarm/next. A stable branch is available at [1] for anyone to pull and resolve potential conflicts.
Thanks!
Signature · 1 line
Kees Cook
Quoted 14 lines · Mostafa Saleh
Many of the sanitizers the kernel supports are disabled when running in EL2 with nvhe/hvhe/proctected modes, some of those are easier (and makes more sense) to integrate than others.
Last year, kCFI support was added in [1]This patchset adds support for UBSAN in EL2.
UBSAN can run in 2 modes:
1) “Normal” (CONFIG_UBSAN_TRAP=n): In this mode the compiler will do the UBSAN checks and insert some function calls in case of failures, it can provide more information(ex: what is the value of the out of bound) about the failures through those function arguments, and those functions(implemented inlib/ubsan.c) will print a report with such errors.
Quoted 1 line
[...]
Applied to next, thanks!
[1/4] arm64: Introduce esr_is_ubsan_brk()
commit: dc1fd37a7f501731e488c1c6f86b2f591632a4ad
[2/4] ubsan: Remove regs from report_ubsan_failure()
commit: d683a8561889c1813fe2ad6082769c91e3cb71b3
[3/4] KVM: arm64: Introduce CONFIG_UBSAN_KVM_EL2
commit: 61b38f7591fb434fce326c1d686a9793c7f418bc
[4/4] KVM: arm64: Handle UBSAN faults
commit: 446692759b0732ef2d9a93b7e6730aa762ccf0abCheers,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.