KVM: arm64: Hypervisor stack enhancements
v4 cover KVM: arm64: Hypervisor stack enhancements
Hi all,
This is v4 of the nVHE hypervisor stack enhancements.
Previous versions can be found at:
v3: https://lore.kernel.org/r/[email protected]/
v2: https://lore.kernel.org/r/[email protected]/
v1: https://lore.kernel.org/r/[email protected]/
The previous cover letter has been copied below for convenience.
Thanks,
Kalesh
-----
This series is based on 5.17-rc5 and adds the following stack features to the KVM nVHE hypervisor:
== Hyp Stack Guard Pages ==
Based on the technique used by arm64 VMAP_STACK to detect overflow. i.e. the stack is aligned to twice its size which ensure that the 'stack shift' bit of any valid SP is 0. The 'stack shift' bit can be tested in the exception entry to detect overflow without corrupting GPRs.
== Hyp Stack Unwinder ==
Based on the arm64 kernel stack unwinder
(See: arch/arm64/kernel/stacktrace.c)
The unwinding and dumping of the hyp stack is not enabled by default and depends on CONFIG_NVHE_EL2_DEBUG to avoid potential information leaks.
When CONFIG_NVHE_EL2_DEBUG is enabled the host stage 2 protection is disabled, allowing the host to read the hypervisor stack pages and unwind the stack from EL1. This allows us to print the hypervisor stacktrace before panicking the host; as shown below.
Example call trace:
[ 98.916444][ T426] kvm [426]: nVHE hyp panic at: [<ffffffc0096156fc>] __kvm_nvhe_overflow_stack+0x8/0x34! [ 98.918360][ T426] nVHE HYP call trace: [ 98.918692][ T426] kvm [426]: [<ffffffc009615aac>] __kvm_nvhe_cpu_prepare_nvhe_panic_info+0x4c/0x68 [ 98.919545][ T426] kvm [426]: [<ffffffc0096159a4>] __kvm_nvhe_hyp_panic+0x2c/0xe8 [ 98.920107][ T426] kvm [426]: [<ffffffc009615ad8>] __kvm_nvhe_hyp_panic_bad_stack+0x10/0x10 [ 98.920665][ T426] kvm [426]: [<ffffffc009610a4c>] __kvm_nvhe___kvm_hyp_host_vector+0x24c/0x794 [ 98.921292][ T426] kvm [426]: [<ffffffc009615718>] __kvm_nvhe_overflow_stack+0x24/0x34
. . .
[ 98.973382][ T426] kvm [426]: [<ffffffc009615718>] __kvm_nvhe_overflow_stack+0x24/0x34 [ 98.973816][ T426] kvm [426]: [<ffffffc0096152f4>] __kvm_nvhe___kvm_vcpu_run+0x38/0x438 [ 98.974255][ T426] kvm [426]: [<ffffffc009616f80>] __kvm_nvhe_handle___kvm_vcpu_run+0x1c4/0x364 [ 98.974719][ T426] kvm [426]: [<ffffffc009616928>] __kvm_nvhe_handle_trap+0xa8/0x130 [ 98.975152][ T426] kvm [426]: [<ffffffc009610064>] __kvm_nvhe___host_exit+0x64/0x64 [ 98.975588][ T426] ---- end of nVHE HYP call trace ----
Kalesh Singh (8):
KVM: arm64: Introduce hyp_alloc_private_va_range()
KVM: arm64: Introduce pkvm_alloc_private_va_range()
KVM: arm64: Add guard pages for KVM nVHE hypervisor stack
KVM: arm64: Add guard pages for pKVM (protected nVHE) hypervisor stack
KVM: arm64: Detect and handle hypervisor stack overflows
KVM: arm64: Add hypervisor overflow stack
KVM: arm64: Unwind and dump nVHE HYP stacktrace
KVM: arm64: Symbolize the nVHE HYP backtrace
arch/arm64/include/asm/kvm_asm.h | 21 +++ arch/arm64/include/asm/kvm_mmu.h | 4 + arch/arm64/include/asm/stacktrace.h | 12 ++ arch/arm64/kernel/stacktrace.c | 210 ++++++++++++++++++++++++--- arch/arm64/kvm/Kconfig | 5 +- arch/arm64/kvm/arm.c | 34 ++++- arch/arm64/kvm/handle_exit.c | 16 +- arch/arm64/kvm/hyp/include/nvhe/mm.h | 3 +- arch/arm64/kvm/hyp/nvhe/host.S | 29 ++++ arch/arm64/kvm/hyp/nvhe/hyp-main.c | 5 +- arch/arm64/kvm/hyp/nvhe/mm.c | 60 +++++--- arch/arm64/kvm/hyp/nvhe/setup.c | 25 +++- arch/arm64/kvm/hyp/nvhe/switch.c | 30 +++- arch/arm64/kvm/mmu.c | 65 ++++++--- scripts/kallsyms.c | 2 +- 15 files changed, 428 insertions(+), 93 deletions(-)
base-commit: cfb92440ee71adcc2105b0890bb01ac3cddb8507
Signature · 1 line
2.35.1.574.g5d30c73bfb-goog
PATCH v4 1/8 KVM: arm64: Introduce hyp_alloc_private_va_range()
hyp_alloc_private_va_range() can be used to reserve private VA ranges in the nVHE hypervisor. Also update __create_hyp_private_mapping() to allow specifying an alignment for the private VA mapping.
These will be used to implement stack guard pages for KVM nVHE hypervisor (nVHE Hyp mode / not pKVM), in a subsequent patch in the series.
---
Changes in v4:
- Handle null ptr in hyp_alloc_private_va_range() and replace IS_ERR_OR_NULL checks in callers with IS_ERR checks, per Fuad
- Fix kernel-doc comments format, per Fuad
Changes in v3:
- Handle null ptr in IS_ERR_OR_NULL checks, per Mark
arch/arm64/include/asm/kvm_mmu.h | 4 ++ arch/arm64/kvm/mmu.c | 65 ++++++++++++++++++++------------ 2 files changed, 45 insertions(+), 24 deletions(-)
arch/arm64/include/asm/kvm_mmu.h +4 -0
@@ -153,6 +153,10 @@ static __always_inline unsigned long __kern_hyp_va(unsigned long v)int kvm_share_hyp (void *from,void *to);void kvm_unshare_hyp (void *from,void *to);int create_hyp_mappings (void *from,void *to,enum kvm_pgtable_prot prot); +unsigned long hyp_alloc_private_va_range (size_t size,size_t align); +int __create_hyp_private_mapping (phys_addr_t phys_addr,size_t size, +size_t align,unsigned long *haddr, +enum kvm_pgtable_prot prot);int create_hyp_io_mappings (phys_addr_t phys_addr,size_t size,void __iomem **kaddr,void __iomem **haddr);
arch/arm64/kvm/mmu.c +41 -24
106 lines not on this page · raw message
Signature · 1 line
2.35.1.574.g5d30c73bfb-goog
PATCH v4 2/8 KVM: arm64: Introduce pkvm_alloc_private_va_range()
In reply to Kalesh Singhpkvm_hyp_alloc_private_va_range() can be used to reserve private VA ranges in the pKVM nVHE hypervisor (). Also update __pkvm_create_private_mapping() to allow specifying an alignment for the private VA mapping.
These will be used to implement stack guard pages for pKVM nVHE hypervisor (in a subsequent patch in the series).
Credits to Quentin Perret <[email protected]> for the idea of moving private VA allocation out of __pkvm_create_private_mapping()
---
Changes in v4:
- Handle null ptr in pkvm_alloc_private_va_range() and replace IS_ERR_OR_NULL checks in callers with IS_ERR checks, per Fuad
- Fix kernel-doc comments format, per Fuad
- Format __pkvm_create_private_mapping() prototype args (< 80 col), per Fuad
Changes in v3:
- Handle null ptr in IS_ERR_OR_NULL checks, per Mark
Changes in v2:
- Allow specifying an alignment for the private VA allocations, per Marc
arch/arm64/kvm/hyp/include/nvhe/mm.h | 3 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 5 ++- arch/arm64/kvm/hyp/nvhe/mm.c | 60 +++++++++++++++++----------- arch/arm64/kvm/mmu.c | 2 +- 4 files changed, 43 insertions(+), 27 deletions(-)
arch/arm64/kvm/hyp/include/nvhe/mm.h +2 -1
@@ -20,7 +20,8 @@ int pkvm_cpu_set_vector(enum arm64_hyp_spectre_vector slot);int pkvm_create_mappings (void *from,void *to,enum kvm_pgtable_prot prot);int pkvm_create_mappings_locked (void *from,void *to,enum kvm_pgtable_prot prot);unsigned long __pkvm_create_private_mapping (phys_addr_t phys,size_t size, -enum kvm_pgtable_prot prot); +size_t align,enum kvm_pgtable_prot prot); +unsigned long pkvm_alloc_private_va_range (size_t size,size_t align);static inline void hyp_vmemmap_range (phys_addr_t phys,unsigned long size,unsigned long *start,unsigned long *end)
arch/arm64/kvm/hyp/nvhe/hyp-main.c +3 -2
@@ -158,9 +158,10 @@ static void handle___pkvm_create_private_mapping(struct kvm_cpu_context *host_ct {DECLARE_REG (phys_addr_t, phys, host_ctxt,1 );DECLARE_REG (size_t, size, host_ctxt,2 ); - DECLARE_REG(enum kvm_pgtable_prot, prot, host_ctxt, 3); +DECLARE_REG (size_t, align, host_ctxt,3 ); + DECLARE_REG(enum kvm_pgtable_prot, prot, host_ctxt, 4); -cpu_reg (host_ctxt,1 ) =__pkvm_create_private_mapping (phys, size, prot); +cpu_reg (host_ctxt,1 ) =__pkvm_create_private_mapping (phys, size, align, prot); }static void handle___pkvm_prot_finalize (struct kvm_cpu_context *host_ctxt)
arch/arm64/kvm/hyp/nvhe/mm.c +37 -23
85 lines not on this page · raw message
arch/arm64/kvm/mmu.c +1 -1
@@ -506,7 +506,7 @@ int __create_hyp_private_mapping(phys_addr_t phys_addr, size_t size,if (!kvm_host_owns_hyp_mappings ()) { addr =kvm_call_hyp_nvhe (__pkvm_create_private_mapping, - phys_addr, size, prot); + phys_addr, size, align, prot);if (IS_ERR ((void *)addr))return PTR_ERR ((void *)addr); *haddr = addr;
Signature · 1 line
2.35.1.574.g5d30c73bfb-goog
Marc Zyngier
maintainerRe: [PATCH v4 2/8] KVM: arm64: Introduce pkvm_alloc_private_va_range()
Quoted 20 lines · Kalesh Singh
pkvm_hyp_alloc_private_va_range()can be used to reserve private VA ranges in the pKVM nVHE hypervisor (). Also update__pkvm_create_private_mapping()to allow specifying an alignment for the private VA mapping.These will be used to implement stack guard pages for pKVM nVHE hypervisor (in a subsequent patch in the series).
Credits to Quentin Perret <[email protected]> for the idea of moving private VA allocation out of
__pkvm_create_private_mapping()Signed-off-by Kalesh Singh <[email protected]>---
Changes in v4:
- Handle null ptr inpkvm_alloc_private_va_range()and replaceIS_ERR_OR_NULLchecks in callers withIS_ERRchecks, per Fuad
- Fix kernel-doc comments format, per Fuad
- Format__pkvm_create_private_mapping()prototype args (< 80 col), per Fuad
Quoted 5 lines
Changes in v3:
- Handle null ptr inIS_ERR_OR_NULLchecks, per MarkChanges in v2:
- Allow specifying an alignment for the private VA allocations, per Marc
I probably badly expressed my earlier concern.
Yes, an alignment is necessary. But how often do we want an alignment that isn't naturally aligned to the size of the allocation (i.e. the power of 2 >= the size of the allocation)? This is what the rest of the kernel does (get_order() and co), and I thing we should follow this.
This applies to both this patch and the previous one.
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v4 2/8] KVM: arm64: Introduce pkvm_alloc_private_va_range()
Quoted 29 lines · Marc Zyngier
Quoted 20 lines · Kalesh Singh
pkvm_hyp_alloc_private_va_range()can be used to reserve private VA ranges in the pKVM nVHE hypervisor (). Also update__pkvm_create_private_mapping()to allow specifying an alignment for the private VA mapping.These will be used to implement stack guard pages for pKVM nVHE hypervisor (in a subsequent patch in the series).
Credits to Quentin Perret <[email protected]> for the idea of moving private VA allocation out of
__pkvm_create_private_mapping()Signed-off-by Kalesh Singh <[email protected]>---
Changes in v4:
- Handle null ptr inpkvm_alloc_private_va_range()and replaceIS_ERR_OR_NULLchecks in callers withIS_ERRchecks, per Fuad
- Fix kernel-doc comments format, per Fuad
- Format__pkvm_create_private_mapping()prototype args (< 80 col), per FuadQuoted 5 lines
Changes in v3:
- Handle null ptr inIS_ERR_OR_NULLchecks, per MarkChanges in v2:
- Allow specifying an alignment for the private VA allocations, per Marc
Quoted 7 lines
I probably badly expressed my earlier concern.
Yes, an alignment is necessary. But how often do we want an alignment that isn't naturally aligned to the size of the allocation (i.e. the power of 2 >= the size of the allocation)? This is what the rest of the kernel does (
get_order()and co), and I thing we should follow this.
Hi Marc,
Thanks for clarifying. I think making the alignment implicitly based on the size here will create unnecessary holes where PAGE_SIZE alignment would be ok and potentially overflow the private VA space earlier. Is it not a concern?
- Kalesh
Quoted 13 lines
This applies to both this patch and the previous one.
Thanks,
M.--
Without deviation from the norm, progress is not possible.--
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
Marc Zyngier
maintainerRe: [PATCH v4 2/8] KVM: arm64: Introduce pkvm_alloc_private_va_range()
Hi Kalesh,
Quoted 39 lines · Kalesh Singh
Quoted 29 lines · Marc Zyngier
Quoted 20 lines · Kalesh Singh
pkvm_hyp_alloc_private_va_range()can be used to reserve private VA ranges in the pKVM nVHE hypervisor (). Also update__pkvm_create_private_mapping()to allow specifying an alignment for the private VA mapping.These will be used to implement stack guard pages for pKVM nVHE hypervisor (in a subsequent patch in the series).
Credits to Quentin Perret <[email protected]> for the idea of moving private VA allocation out of
__pkvm_create_private_mapping()Signed-off-by Kalesh Singh <[email protected]>---
Changes in v4:
- Handle null ptr inpkvm_alloc_private_va_range()and replaceIS_ERR_OR_NULLchecks in callers withIS_ERRchecks, per Fuad
- Fix kernel-doc comments format, per Fuad
- Format__pkvm_create_private_mapping()prototype args (< 80 col), per FuadQuoted 5 lines
Changes in v3:
- Handle null ptr inIS_ERR_OR_NULLchecks, per MarkChanges in v2:
- Allow specifying an alignment for the private VA allocations, per MarcQuoted 7 lines
I probably badly expressed my earlier concern.
Yes, an alignment is necessary. But how often do we want an alignment that isn't naturally aligned to the size of the allocation (i.e. the power of 2 >= the size of the allocation)? This is what the rest of the kernel does (
get_order()and co), and I thing we should follow this.
Quoted 6 lines
Hi Marc,
Thanks for clarifying. I think making the alignment implicitly based on the size here will create unnecessary holes where
PAGE_SIZEalignment would be ok and potentially overflow the private VA space earlier. Is it not a concern?
I don't think we should worry too much about this. Even when building the kernel with a very small VA space (commonly 39 bits), we still have a quarter of that reserved for private EL2 mappings. That's pretty big.
We will use a bit more of the memory that is set aside for EL2 page tables, but this shouldn't be a problem either.
Thanks,
M.
Signature · 5 lines
Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH v4 2/8] KVM: arm64: Introduce pkvm_alloc_private_va_range()
Quoted 51 lines · Marc Zyngier
Hi Kalesh,
Quoted 39 lines · Kalesh Singh
Quoted 29 lines · Marc Zyngier
Quoted 20 lines · Kalesh Singh
pkvm_hyp_alloc_private_va_range()can be used to reserve private VA ranges in the pKVM nVHE hypervisor (). Also update__pkvm_create_private_mapping()to allow specifying an alignment for the private VA mapping.These will be used to implement stack guard pages for pKVM nVHE hypervisor (in a subsequent patch in the series).
Credits to Quentin Perret <[email protected]> for the idea of moving private VA allocation out of
__pkvm_create_private_mapping()Signed-off-by Kalesh Singh <[email protected]>---
Changes in v4:
- Handle null ptr inpkvm_alloc_private_va_range()and replaceIS_ERR_OR_NULLchecks in callers withIS_ERRchecks, per Fuad
- Fix kernel-doc comments format, per Fuad
- Format__pkvm_create_private_mapping()prototype args (< 80 col), per FuadQuoted 5 lines
Changes in v3:
- Handle null ptr inIS_ERR_OR_NULLchecks, per MarkChanges in v2:
- Allow specifying an alignment for the private VA allocations, per MarcQuoted 7 lines
I probably badly expressed my earlier concern.
Yes, an alignment is necessary. But how often do we want an alignment that isn't naturally aligned to the size of the allocation (i.e. the power of 2 >= the size of the allocation)? This is what the rest of the kernel does (
get_order()and co), and I thing we should follow this.Quoted 6 lines
Hi Marc,
Thanks for clarifying. I think making the alignment implicitly based on the size here will create unnecessary holes where
PAGE_SIZEalignment would be ok and potentially overflow the private VA space earlier. Is it not a concern?
Quoted 7 lines
I don't think we should worry too much about this. Even when building the kernel with a very small VA space (commonly 39 bits), we still have a quarter of that reserved for private EL2 mappings. That's pretty big.
We will use a bit more of the memory that is set aside for EL2 page tables, but this shouldn't be a problem either.
Hi Marc,
Thanks for the explanations. I'll update as suggested in the next version.
- Kalesh
Quoted 7 lines
Thanks,
M.--
Without deviation from the norm, progress is not possible.
_______________________________________________
kvmarm mailing list
[email protected]
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v4 3/8 KVM: arm64: Add guard pages for KVM nVHE hypervisor stack
In reply to Kalesh SinghMaps the stack pages in the flexible private VA range and allocates guard pages below the stack as unbacked VA space. The stack is aligned to twice its size to aid overflow detection (implemented in a subsequent patch in the series).
---
Changes in v4:
- Replace IS_ERR_OR_NULL check with IS_ERR check now that hyp_alloc_private_va_range() returns an error for null pointer, per Fuad
- Format comments to < 80 cols, per Fuad
Changes in v3:
- Handle null ptr in IS_ERR_OR_NULL checks, per Mark
arch/arm64/include/asm/kvm_asm.h | 1 + arch/arm64/kvm/arm.c | 32 +++++++++++++++++++++++++++++--- 2 files changed, 30 insertions(+), 3 deletions(-)
arch/arm64/include/asm/kvm_asm.h +1 -0
@@ -169,6 +169,7 @@ struct kvm_nvhe_init_params {unsigned long tcr_el2;unsigned long tpidr_el2;unsigned long stack_hyp_va; +unsigned long stack_pa;phys_addr_t pgd_pa;unsigned long hcr_el2;unsigned long vttbr;
arch/arm64/kvm/arm.c +29 -3
@@ -1541,7 +1541,6 @@ static void cpu_prepare_hyp_mode(int cpu) tcr |= (idmap_t0sz &GENMASK (TCR_TxSZ_WIDTH -1 ,0 )) <<TCR_T0SZ_OFFSET ; params->tcr_el2 = tcr; - params->stack_hyp_va =kern_hyp_va (per_cpu (kvm_arm_hyp_stack_page, cpu) +PAGE_SIZE ); params->pgd_pa =kvm_mmu_get_httbr ();if (is_protected_kvm_enabled ()) params->hcr_el2 =HCR_HOST_NVHE_PROTECTED_FLAGS ;
@@ -1990,14 +1989,41 @@ static int init_hyp_mode(void)* Map the Hyp stack pages */ for_each_possible_cpu (cpu) { +struct kvm_nvhe_init_params *params =per_cpu_ptr_nvhe_sym (kvm_init_params, cpu);char *stack_page = (char *)per_cpu (kvm_arm_hyp_stack_page, cpu); - err =create_hyp_mappings (stack_page, stack_page +PAGE_SIZE , -PAGE_HYP ); +unsigned long stack_hyp_va, guard_hyp_va; +/* +* Private mappings are allocated downwards from io_map_base +* so allocate the stack first then the guard page. +* +* The stack is aligned to twice its size to facilitate overflow +* detection. +*/ + err =__create_hyp_private_mapping (__pa (stack_page),PAGE_SIZE , +PAGE_SIZE *2 , &stack_hyp_va,PAGE_HYP );if (err) {kvm_err ("Cannot map hyp stack\n" ); goto out_err; } + +/* Allocate unbacked private VA range for stack guard page */ + guard_hyp_va =hyp_alloc_private_va_range (PAGE_SIZE ,PAGE_SIZE ); +if (IS_ERR ((void *)guard_hyp_va)) { + err =PTR_ERR ((void *)guard_hyp_va); +kvm_err ("Cannot allocate hyp stack guard page\n" ); + goto out_err; + } + +/* +* Save the stack PA in nvhe_init_params. This will be needed +* to recreate the stack mapping in protected nVHE mode. +* __hyp_pa() won't do the right thing there, since the stack +* has been mapped in the flexible private VA space. +*/ + params->stack_pa =__pa (stack_page) +PAGE_SIZE ; + + params->stack_hyp_va = stack_hyp_va +PAGE_SIZE ; }for_each_possible_cpu (cpu) {
Signature · 1 line
2.35.1.574.g5d30c73bfb-goog
Marc Zyngier
maintainerRe: [PATCH v4 3/8] KVM: arm64: Add guard pages for KVM nVHE hypervisor stack
Quoted 55 lines · Kalesh Singh
Maps the stack pages in the flexible private VA range and allocates guard pages below the stack as unbacked VA space. The stack is aligned to twice its size to aid overflow detection (implemented in a subsequent patch in the series).
Signed-off-by Kalesh Singh <[email protected]>---
Changes in v4:
- ReplaceIS_ERR_OR_NULLcheck withIS_ERRcheck now thathyp_alloc_private_va_range()returns an error for null pointer, per Fuad
- Format comments to < 80 cols, per FuadChanges in v3:
- Handle null ptr inIS_ERR_OR_NULLchecks, per Markarch/arm64/include/asm/kvm_asm.h | 1 + arch/arm64/kvm/arm.c | 32 +++++++++++++++++++++++++++++--- 2 files changed, 30 insertions(+), 3 deletions(-)arch/arm64/include/asm/kvm_asm.h +1
-0@@ -169,6 +169,7 @@ struct kvm_nvhe_init_params {unsigned long tcr_el2;unsigned long tpidr_el2;unsigned long stack_hyp_va; +unsigned long stack_pa;phys_addr_t pgd_pa;unsigned long hcr_el2;unsigned long vttbr;arch/arm64/kvm/arm.c +2
-3@@ -1541,7 +1541,6 @@ static void cpu_prepare_hyp_mode(int cpu) tcr |= (idmap_t0sz &GENMASK (TCR_TxSZ_WIDTH -1 ,0 )) <<TCR_T0SZ_OFFSET ; params->tcr_el2 = tcr; - params->stack_hyp_va =kern_hyp_va (per_cpu (kvm_arm_hyp_stack_page, cpu) +PAGE_SIZE ); params->pgd_pa =kvm_mmu_get_httbr ();if (is_protected_kvm_enabled ()) params->hcr_el2 =HCR_HOST_NVHE_PROTECTED_FLAGS ;@@ -1990,14 +1989,41 @@ static int init_hyp_mode(void)* Map the Hyp stack pages */ for_each_possible_cpu (cpu) { +struct kvm_nvhe_init_params *params =per_cpu_ptr_nvhe_sym (kvm_init_params, cpu);char *stack_page = (char *)per_cpu (kvm_arm_hyp_stack_page, cpu); - err =create_hyp_mappings (stack_page, stack_page +PAGE_SIZE , -PAGE_HYP ); +unsigned long stack_hyp_va, guard_hyp_va;
arch/arm64/kvm/arm.c static int init_hyp_mode(void) · line 2004
Quoted 10 lines
arch/arm64/kvm/arm.c +9
-0+/* +* Private mappings are allocated downwards from io_map_base +* so allocate the stack first then the guard page. +* +* The stack is aligned to twice its size to facilitate overflow +* detection. +*/ + err =__create_hyp_private_mapping (__pa (stack_page),PAGE_SIZE , +PAGE_SIZE *2 , &stack_hyp_va,PAGE_HYP );
Right, I guess that's where my earlier ask breaks, as you want an alignment that is *larger* than the allocation.
arch/arm64/kvm/arm.c static int init_hyp_mode(void) · line 2011
Quoted 7 lines
arch/arm64/kvm/arm.c +3
-0if (err) {kvm_err ("Cannot map hyp stack\n" ); goto out_err; } + +/* Allocate unbacked private VA range for stack guard page */ + guard_hyp_va =hyp_alloc_private_va_range (PAGE_SIZE ,PAGE_SIZE );
Huh. You are implicitly relying on the VA allocator handing you an address contiguous with the previous mapping. That's... brave. I'd rather you allocate the VA space upfront with the correct alignment and then map the single page where it should be in the VA region.
That'd be a lot less fragile.
Quoted 8 lines
arch/arm64/kvm/arm.c +8
-0+if (IS_ERR ((void *)guard_hyp_va)) { + err =PTR_ERR ((void *)guard_hyp_va); +kvm_err ("Cannot allocate hyp stack guard page\n" ); + goto out_err; + } + +/* +* Save the stack PA in nvhe_init_params. This will be needed
arch/arm64/kvm/arm.c static int init_hyp_mode(void) · line 2029
Quoted 10 lines
arch/arm64/kvm/arm.c +7
-0+* to recreate the stack mapping in protected nVHE mode. +* __hyp_pa() won't do the right thing there, since the stack +* has been mapped in the flexible private VA space. +*/ + params->stack_pa =__pa (stack_page) +PAGE_SIZE ; + + params->stack_hyp_va = stack_hyp_va +PAGE_SIZE ; }for_each_possible_cpu (cpu) {
Thanks,
M.
Signature · 5 lines
Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH v4 3/8] KVM: arm64: Add guard pages for KVM nVHE hypervisor stack
Quoted 80 lines · Marc Zyngier
Quoted 55 lines · Kalesh Singh
Maps the stack pages in the flexible private VA range and allocates guard pages below the stack as unbacked VA space. The stack is aligned to twice its size to aid overflow detection (implemented in a subsequent patch in the series).
Signed-off-by Kalesh Singh <[email protected]>---
Changes in v4:
- ReplaceIS_ERR_OR_NULLcheck withIS_ERRcheck now thathyp_alloc_private_va_range()returns an error for null pointer, per Fuad
- Format comments to < 80 cols, per FuadChanges in v3:
- Handle null ptr inIS_ERR_OR_NULLchecks, per Markarch/arm64/include/asm/kvm_asm.h | 1 + arch/arm64/kvm/arm.c | 32 +++++++++++++++++++++++++++++--- 2 files changed, 30 insertions(+), 3 deletions(-)arch/arm64/include/asm/kvm_asm.h +1
-0@@ -169,6 +169,7 @@ struct kvm_nvhe_init_params {unsigned long tcr_el2;unsigned long tpidr_el2;unsigned long stack_hyp_va; +unsigned long stack_pa;phys_addr_t pgd_pa;unsigned long hcr_el2;unsigned long vttbr;arch/arm64/kvm/arm.c +2
-3@@ -1541,7 +1541,6 @@ static void cpu_prepare_hyp_mode(int cpu) tcr |= (idmap_t0sz &GENMASK (TCR_TxSZ_WIDTH -1 ,0 )) <<TCR_T0SZ_OFFSET ; params->tcr_el2 = tcr; - params->stack_hyp_va =kern_hyp_va (per_cpu (kvm_arm_hyp_stack_page, cpu) +PAGE_SIZE ); params->pgd_pa =kvm_mmu_get_httbr ();if (is_protected_kvm_enabled ()) params->hcr_el2 =HCR_HOST_NVHE_PROTECTED_FLAGS ;@@ -1990,14 +1989,41 @@ static int init_hyp_mode(void)* Map the Hyp stack pages */ for_each_possible_cpu (cpu) { +struct kvm_nvhe_init_params *params =per_cpu_ptr_nvhe_sym (kvm_init_params, cpu);char *stack_page = (char *)per_cpu (kvm_arm_hyp_stack_page, cpu); - err =create_hyp_mappings (stack_page, stack_page +PAGE_SIZE , -PAGE_HYP ); +unsigned long stack_hyp_va, guard_hyp_va;arch/arm64/kvm/arm.c static int init_hyp_mode(void) · line 2004
Quoted 10 lines
arch/arm64/kvm/arm.c +9
-0+/* +* Private mappings are allocated downwards from io_map_base +* so allocate the stack first then the guard page. +* +* The stack is aligned to twice its size to facilitate overflow +* detection. +*/ + err =__create_hyp_private_mapping (__pa (stack_page),PAGE_SIZE , +PAGE_SIZE *2 , &stack_hyp_va,PAGE_HYP );Right, I guess that's where my earlier ask breaks, as you want an alignment that is *larger* than the allocation.
arch/arm64/kvm/arm.c static int init_hyp_mode(void) · line 2011
Quoted 7 lines
arch/arm64/kvm/arm.c +3
-0if (err) {kvm_err ("Cannot map hyp stack\n" ); goto out_err; } + +/* Allocate unbacked private VA range for stack guard page */ + guard_hyp_va =hyp_alloc_private_va_range (PAGE_SIZE ,PAGE_SIZE );
Quoted 6 lines
Huh. You are implicitly relying on the VA allocator handing you an address contiguous with the previous mapping. That's... brave. I'd rather you allocate the VA space upfront with the correct alignment and then map the single page where it should be in the VA region.
That'd be a lot less fragile.
Agreed. I'll fix it in the next version.
Thanks,
Kalesh
Quoted 26 lines
Quoted 9 lines
arch/arm64/kvm/arm.c +8
-0+if (IS_ERR ((void *)guard_hyp_va)) { + err =PTR_ERR ((void *)guard_hyp_va); +kvm_err ("Cannot allocate hyp stack guard page\n" ); + goto out_err; + } + +/* +* Save the stack PA in nvhe_init_params. This will be needed arch/arm64/kvm/arm.c static int init_hyp_mode(void) · line 2029
Quoted 10 lines
arch/arm64/kvm/arm.c +7
-0+* to recreate the stack mapping in protected nVHE mode. +* __hyp_pa() won't do the right thing there, since the stack +* has been mapped in the flexible private VA space. +*/ + params->stack_pa =__pa (stack_page) +PAGE_SIZE ; + + params->stack_hyp_va = stack_hyp_va +PAGE_SIZE ; }for_each_possible_cpu (cpu) {Thanks,
M.--
Without deviation from the norm, progress is not possible.
PATCH v4 4/8 KVM: arm64: Add guard pages for pKVM (protected nVHE) hypervisor stack
In reply to Kalesh SinghMaps the stack pages in the flexible private VA range and allocates guard pages below the stack as unbacked VA space. The stack is aligned to twice its size to aid overflow detection (implemented in a subsequent patch in the series).
---
Changes in v4:
- Replace IS_ERR_OR_NULL check with IS_ERR check now that pkvm_alloc_private_va_range() returns an error for null pointer, per Fuad
Changes in v3:
- Handle null ptr in IS_ERR_OR_NULL checks, per Mark
arch/arm64/kvm/hyp/nvhe/setup.c | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-)
arch/arm64/kvm/hyp/nvhe/setup.c +21 -4
@@ -105,11 +105,28 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size,if (ret)return ret; - end = (void *)per_cpu_ptr (&kvm_init_params, i)->stack_hyp_va; +/* +* Private mappings are allocated upwards from __io_map_base +* so allocate the guard page first then the stack. +*/ + start = (void *)pkvm_alloc_private_va_range (PAGE_SIZE ,PAGE_SIZE ); +if (IS_ERR (start)) +return PTR_ERR (start); + +/* +* The stack is aligned to twice its size to facilitate overflow +* detection. +*/ + end = (void *)per_cpu_ptr (&kvm_init_params, i)->stack_pa; start = end -PAGE_SIZE ; - ret =pkvm_create_mappings (start, end,PAGE_HYP ); -if (ret) -return ret; + start = (void *)__pkvm_create_private_mapping ((phys_addr_t )start, +PAGE_SIZE ,PAGE_SIZE *2 ,PAGE_HYP ); +if (IS_ERR (start)) +return PTR_ERR (start); + end = start +PAGE_SIZE ; + +/* Update stack_hyp_va to end of the stack's private VA range */ +per_cpu_ptr (&kvm_init_params, i)->stack_hyp_va = (unsigned long ) end; }/*
Signature · 1 line
2.35.1.574.g5d30c73bfb-goog
Marc Zyngier
maintainerRe: [PATCH v4 4/8] KVM: arm64: Add guard pages for pKVM (protected nVHE) hypervisor stack
Quoted 38 lines · Kalesh Singh
Maps the stack pages in the flexible private VA range and allocates guard pages below the stack as unbacked VA space. The stack is aligned to twice its size to aid overflow detection (implemented in a subsequent patch in the series).
Signed-off-by Kalesh Singh <[email protected]>---
Changes in v4:
- ReplaceIS_ERR_OR_NULLcheck withIS_ERRcheck now thatpkvm_alloc_private_va_range()returns an error for null pointer, per FuadChanges in v3:
- Handle null ptr inIS_ERR_OR_NULLchecks, per Markarch/arm64/kvm/hyp/nvhe/setup.c | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-)arch/arm64/kvm/hyp/nvhe/setup.c +9
-1@@ -105,11 +105,28 @@ static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size,if (ret)return ret; - end = (void *)per_cpu_ptr (&kvm_init_params, i)->stack_hyp_va; +/* +* Private mappings are allocated upwards from __io_map_base +* so allocate the guard page first then the stack. +*/ + start = (void *)pkvm_alloc_private_va_range (PAGE_SIZE ,PAGE_SIZE ); +if (IS_ERR (start)) +return PTR_ERR (start); + +/*
arch/arm64/kvm/hyp/nvhe/setup.c static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, · line 123
Quoted 10 lines
arch/arm64/kvm/hyp/nvhe/setup.c +6
-3+* The stack is aligned to twice its size to facilitate overflow +* detection. +*/ + end = (void *)per_cpu_ptr (&kvm_init_params, i)->stack_pa; start = end -PAGE_SIZE ; - ret =pkvm_create_mappings (start, end,PAGE_HYP ); -if (ret) -return ret; + start = (void *)__pkvm_create_private_mapping ((phys_addr_t )start, +PAGE_SIZE ,PAGE_SIZE *2 ,PAGE_HYP );
Similar comments as the previous patch. I'd rather you treat each stack as a two-page VA, populated by a single page. It would be a lot clearer, and less fragile.
arch/arm64/kvm/hyp/nvhe/setup.c static int recreate_hyp_mappings(phys_addr_t phys, unsigned long size, · line 132
Quoted 9 lines
arch/arm64/kvm/hyp/nvhe/setup.c +6
-0+if (IS_ERR (start)) +return PTR_ERR (start); + end = start +PAGE_SIZE ; + +/* Update stack_hyp_va to end of the stack's private VA range */ +per_cpu_ptr (&kvm_init_params, i)->stack_hyp_va = (unsigned long ) end; }/*
Thanks,
M.
Signature · 5 lines
Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list [email protected] https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
PATCH v4 5/8 KVM: arm64: Detect and handle hypervisor stack overflows
In reply to Kalesh SinghThe hypervisor stacks (for both nVHE Hyp mode and nVHE protected mode) are aligned to twice their size (PAGE_SIZE), meaning that any valid stack address has PAGE_SHIFT bit as 0. This allows us to conveniently check for overflow in the exception entry without corrupting any GPRs. We won't recover from a stack overflow so panic the hypervisor.
---
Changes in v3:
- Remove test_sp_overflow macro, per Mark
- Add asmlinkage attribute for hyp_panic, hyp_panic_bad_stack, per Ard
arch/arm64/kvm/hyp/nvhe/host.S | 24 ++++++++++++++++++++++++ arch/arm64/kvm/hyp/nvhe/switch.c | 7 ++++++- 2 files changed, 30 insertions(+), 1 deletion(-)
arch/arm64/kvm/hyp/nvhe/host.S +24 -0
@@ -153,6 +153,18 @@ SYM_FUNC_END(__host_hvc) .macro invalid_host_el2_vect .align 7 + + /* + * Test whether the SP has overflowed, without corrupting a GPR. + * nVHE hypervisor stacks are aligned so that SP & (1 << PAGE_SHIFT) + * should always be zero. + */ + add sp, sp, x0 // sp' = sp + x0 + sub x0, sp, x0 // x0' = sp' - x0 = (sp + x0) - x0 = sp + tbnz x0, #PAGE_SHIFT, .L__hyp_sp_overflow\@ + sub x0, sp, x0 // x0'' = sp' - x0' = (sp + x0) - sp = x0 + sub sp, sp, x0 // sp'' = sp' - x0 = (sp + x0) - x0 = sp + /* If a guest is loaded, panic out of it. */ stp x0, x1, [sp, #-16]! get_loaded_vcpu x0, x1
@@ -165,6 +177,18 @@ SYM_FUNC_END(__host_hvc) * been partially clobbered by __host_enter. */ b hyp_panic + +.L__hyp_sp_overflow\@: + /* + * Reset SP to the top of the stack, to allow handling the hyp_panic. + * This corrupts the stack but is ok, since we won't be attempting + * any unwinding here. + */ + ldr_this_cpu x0, kvm_init_params + NVHE_INIT_STACK_HYP_VA, x1 + mov sp, x0 + + bl hyp_panic_bad_stack + ASM_BUG() .endm .macro invalid_host_el1_vect
arch/arm64/kvm/hyp/nvhe/switch.c +6 -1
@@ -347,7 +347,7 @@ int __kvm_vcpu_run(struct kvm_vcpu *vcpu)return exit_code; } -void __noreturn hyp_panic (void ) +asmlinkage void __noreturn hyp_panic(void) {u64 spsr =read_sysreg_el2 (SYS_SPSR );u64 elr =read_sysreg_el2 (SYS_ELR );
@@ -369,6 +369,11 @@ void __noreturn hyp_panic(void)unreachable (); } +asmlinkage void __noreturn hyp_panic_bad_stack(void) +{ +hyp_panic (); +} + asmlinkage void kvm_unexpected_el2_exception(void) {return __kvm_unexpected_el2_exception ();
Signature · 1 line
2.35.1.574.g5d30c73bfb-goog
PATCH v4 6/8 KVM: arm64: Add hypervisor overflow stack
In reply to Kalesh SinghAllocate and switch to 16-byte aligned secondary stack on overflow. This provides us stack space to better handle overflows; and is used in a subsequent patch to dump the hypervisor stacktrace. The overflow stack is only allocated if CONFIG_NVHE_EL2_DEBUG is enabled, as hypervisor stacktraces is a debug feature dependent on CONFIG_NVHE_EL2_DEBUG.
---
Changes in v4:
- Update comment to clarify resetting the SP to the top of the stack only happens if CONFIG_NVHE_EL2_DEBUG is disabled, per Fuad
arch/arm64/kvm/hyp/nvhe/host.S | 11 ++++++++--- arch/arm64/kvm/hyp/nvhe/switch.c | 5 +++++ 2 files changed, 13 insertions(+), 3 deletions(-)
arch/arm64/kvm/hyp/nvhe/host.S +8 -3
@@ -179,13 +179,18 @@ SYM_FUNC_END(__host_hvc) b hyp_panic .L__hyp_sp_overflow\@: +#ifdef CONFIG_NVHE_EL2_DEBUG + /* Switch to the overflow stack */ + adr_this_cpu sp, hyp_overflow_stack + PAGE_SIZE, x0 +#else /* - * Reset SP to the top of the stack, to allow handling the hyp_panic. - * This corrupts the stack but is ok, since we won't be attempting - * any unwinding here. + * If !CONFIG_NVHE_EL2_DEBUG, reset SP to the top of the stack, to + * allow handling the hyp_panic. This corrupts the stack but is ok, + * since we won't be attempting any unwinding here. */ ldr_this_cpu x0, kvm_init_params + NVHE_INIT_STACK_HYP_VA, x1 mov sp, x0 +#endif bl hyp_panic_bad_stack ASM_BUG()
arch/arm64/kvm/hyp/nvhe/switch.c +5 -0
@@ -34,6 +34,11 @@ DEFINE_PER_CPU(struct kvm_host_data, kvm_host_data); DEFINE_PER_CPU(struct kvm_cpu_context, kvm_hyp_ctxt); DEFINE_PER_CPU(unsigned long, kvm_hyp_vector); +#ifdef CONFIG_NVHE_EL2_DEBUG +DEFINE_PER_CPU(unsigned long [PAGE_SIZE/sizeof(long)], hyp_overflow_stack) +__aligned(16) ; +#endif +static void __activate_traps (struct kvm_vcpu *vcpu) {u64 val;
Signature · 1 line
2.35.1.574.g5d30c73bfb-goog
PATCH v4 7/8 KVM: arm64: Unwind and dump nVHE HYP stacktrace
In reply to Kalesh SinghUnwind the stack in EL1, when CONFIG_NVHE_EL2_DEBUG is enabled. This is possible because CONFIG_NVHE_EL2_DEBUG disables the host stage-2 protection on hyp_panic(), allowing the host to access the hypervisor stack pages in EL1.
A simple stack overflow test produces the following output:
[ 580.376051][ T412] kvm: nVHE hyp panic at: ffffffc0116145c4! [ 580.378034][ T412] kvm [412]: nVHE HYP call trace: [ 580.378591][ T412] kvm [412]: [<ffffffc011614934>] [ 580.378993][ T412] kvm [412]: [<ffffffc01160fa48>] [ 580.379386][ T412] kvm [412]: [<ffffffc0116145dc>] // Non-terminating recursive call [ 580.379772][ T412] kvm [412]: [<ffffffc0116145dc>] [ 580.380158][ T412] kvm [412]: [<ffffffc0116145dc>] [ 580.380544][ T412] kvm [412]: [<ffffffc0116145dc>] [ 580.380928][ T412] kvm [412]: [<ffffffc0116145dc>]
. . .
Since nVHE hyp symbols are not included by kallsyms to avoid issues with aliasing, we fallback to the vmlinux addresses. Symbolizing the addresses is handled in the next patch in this series.
---
Changes in v4:
- Update commit text and struct kvm_nvhe_panic_info kernel-doc comment to clarify that CONFIG_NVHE_EL2_DEBUG only disables the host stage-2 protection on hyp_panic(), per Fuad
- Update NVHE_EL2_DEBUG Kconfig description to clarify that the hypervisor stack trace is printed when hyp_panic() is called, per Fuad
Changes in v3:
- The nvhe hyp stack unwinder now makes use of the core logic from the regular kernel unwinder to avoid duplication, per Mark
Changes in v2:
- Add cpu_prepare_nvhe_panic_info()
- Move updating the panic info to hyp_panic(), so that unwinding also works for conventional nVHE Hyp-mode.
arch/arm64/include/asm/kvm_asm.h | 20 +++ arch/arm64/include/asm/stacktrace.h | 12 ++ arch/arm64/kernel/stacktrace.c | 210 +++++++++++++++++++++++++--- arch/arm64/kvm/Kconfig | 5 +- arch/arm64/kvm/arm.c | 2 +- arch/arm64/kvm/handle_exit.c | 3 + arch/arm64/kvm/hyp/nvhe/switch.c | 18 +++ 7 files changed, 244 insertions(+), 26 deletions(-)
arch/arm64/include/asm/kvm_asm.h +20 -0
@@ -176,6 +176,26 @@ struct kvm_nvhe_init_params {unsigned long vtcr; }; +#ifdef CONFIG_NVHE_EL2_DEBUG +/** +* struct kvm_nvhe_panic_info - nVHE hypervisor panic info. +* @hyp_stack_base: hyp VA of the hyp_stack base. +* @hyp_overflow_stack_base: hyp VA of the hyp_overflow_stack base. +* @fp: hyp FP where the backtrace begins. +* @pc: hyp PC where the backtrace begins. +* +* Used by the host in EL1 to dump the nVHE hypervisor backtrace on +* hyp_panic. This is possible because CONFIG_NVHE_EL2_DEBUG disables +* the host stage 2 protection on hyp_panic(). See: __hyp_do_panic() +*/ +struct kvm_nvhe_panic_info { +unsigned long hyp_stack_base; +unsigned long hyp_overflow_stack_base; +unsigned long fp; +unsigned long pc; +}; +#endif /* CONFIG_NVHE_EL2_DEBUG */ +/* Translate a kernel address @ptr into its equivalent linear mapping */ #define kvm_ksym_ref (ptr) \ ({ \
arch/arm64/include/asm/stacktrace.h +12 -0
@@ -22,6 +22,10 @@ enum stack_type {STACK_TYPE_OVERFLOW ,STACK_TYPE_SDEI_NORMAL ,STACK_TYPE_SDEI_CRITICAL , +#ifdef CONFIG_NVHE_EL2_DEBUG +STACK_TYPE_KVM_NVHE_HYP , +STACK_TYPE_KVM_NVHE_OVERFLOW , +#endif /* CONFIG_NVHE_EL2_DEBUG */ __NR_STACK_TYPES };
@@ -147,4 +151,12 @@ static inline bool on_accessible_stack(const struct task_struct *tsk, return false; } +#ifdef CONFIG_NVHE_EL2_DEBUG +void kvm_nvhe_dump_backtrace (unsigned long hyp_offset); +#else +static inline void kvm_nvhe_dump_backtrace (unsigned long hyp_offset) +{ +} +#endif /* CONFIG_NVHE_EL2_DEBUG */ +#endif /* __ASM_STACKTRACE_H */
arch/arm64/kernel/stacktrace.c +187 -23
270 lines not on this page · raw message
arch/arm64/kvm/Kconfig +3 -2
@@ -51,8 +51,9 @@ config NVHE_EL2_DEBUG depends on KVM help Say Y here to enable the debug mode for the non-VHE KVM EL2 object. - Failure reports will BUG() in the hypervisor. This is intended for - local EL2 hypervisor development. + Failure reports will BUG() in the hypervisor; and calls to hyp_panic() + will result in printing the hypervisor call stack. + This is intended for local EL2 hypervisor development. If unsure, say N.
arch/arm64/kvm/arm.c +1 -1
@@ -49,7 +49,7 @@ DEFINE_STATIC_KEY_FALSE(kvm_protected_mode_initialized); DECLARE_KVM_HYP_PER_CPU(unsigned long, kvm_hyp_vector); -static DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page); +DEFINE_PER_CPU(unsigned long, kvm_arm_hyp_stack_page);unsigned long kvm_arm_hyp_percpu_base[NR_CPUS ]; DECLARE_KVM_NVHE_PER_CPU(struct kvm_nvhe_init_params, kvm_init_params);
arch/arm64/kvm/handle_exit.c +3 -0
@@ -17,6 +17,7 @@#include <asm/kvm_emulate.h> #include <asm/kvm_mmu.h> #include <asm/debug-monitors.h> +#include <asm/stacktrace.h> #include <asm/traps.h> #include <kvm/arm_hypercalls.h>
@@ -326,6 +327,8 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr,kvm_err ("nVHE hyp panic at: %016llx!\n" , elr_virt + hyp_offset); } +kvm_nvhe_dump_backtrace (hyp_offset); +/* * Hyp has panicked and we're going to handle that by panicking the * kernel. The kernel offset will be revealed in the panic so we're
arch/arm64/kvm/hyp/nvhe/switch.c +18 -0
@@ -37,6 +37,22 @@ DEFINE_PER_CPU(unsigned long, kvm_hyp_vector);#ifdef CONFIG_NVHE_EL2_DEBUG DEFINE_PER_CPU(unsigned long [PAGE_SIZE/sizeof(long)], hyp_overflow_stack)__aligned(16) ; +DEFINE_PER_CPU(struct kvm_nvhe_panic_info, kvm_panic_info); + +static inline void cpu_prepare_nvhe_panic_info (void ) +{ +struct kvm_nvhe_panic_info *panic_info =this_cpu_ptr (&kvm_panic_info); +struct kvm_nvhe_init_params *params =this_cpu_ptr (&kvm_init_params); + + panic_info->hyp_stack_base = (unsigned long )(params->stack_hyp_va -PAGE_SIZE ); + panic_info->hyp_overflow_stack_base = (unsigned long )this_cpu_ptr (hyp_overflow_stack); + panic_info->fp = (unsigned long )__builtin_frame_address (0 ); + panic_info->pc = _THIS_IP_; +} +#else +static inline void cpu_prepare_nvhe_panic_info (void ) +{ +}#endif static void __activate_traps (struct kvm_vcpu *vcpu)
@@ -360,6 +376,8 @@ asmlinkage void __noreturn hyp_panic(void)struct kvm_cpu_context *host_ctxt;struct kvm_vcpu *vcpu; +cpu_prepare_nvhe_panic_info (); + host_ctxt = &this_cpu_ptr (&kvm_host_data)->host_ctxt; vcpu = host_ctxt->__hyp_running_vcpu;
Signature · 1 line
2.35.1.574.g5d30c73bfb-goog
PATCH v4 8/8 KVM: arm64: Symbolize the nVHE HYP backtrace
In reply to Kalesh SinghReintroduce the __kvm_nvhe_ symbols in kallsyms, ignoring the local symbols in this namespace. The local symbols are not informative and can cause aliasing issues when symbolizing the addresses.
With the necessary symbols now in kallsyms we can symbolize nVHE stacktrace addresses using the %pB print format specifier.
Example call trace:
[ 98.916444][ T426] kvm [426]: nVHE hyp panic at: [<ffffffc0096156fc>] __kvm_nvhe_overflow_stack+0x8/0x34! [ 98.918360][ T426] nVHE HYP call trace: [ 98.918692][ T426] kvm [426]: [<ffffffc009615aac>] __kvm_nvhe_cpu_prepare_nvhe_panic_info+0x4c/0x68 [ 98.919545][ T426] kvm [426]: [<ffffffc0096159a4>] __kvm_nvhe_hyp_panic+0x2c/0xe8 [ 98.920107][ T426] kvm [426]: [<ffffffc009615ad8>] __kvm_nvhe_hyp_panic_bad_stack+0x10/0x10 [ 98.920665][ T426] kvm [426]: [<ffffffc009610a4c>] __kvm_nvhe___kvm_hyp_host_vector+0x24c/0x794 [ 98.921292][ T426] kvm [426]: [<ffffffc009615718>] __kvm_nvhe_overflow_stack+0x24/0x34
. . .
[ 98.973382][ T426] kvm [426]: [<ffffffc009615718>] __kvm_nvhe_overflow_stack+0x24/0x34 [ 98.973816][ T426] kvm [426]: [<ffffffc0096152f4>] __kvm_nvhe___kvm_vcpu_run+0x38/0x438 [ 98.974255][ T426] kvm [426]: [<ffffffc009616f80>] __kvm_nvhe_handle___kvm_vcpu_run+0x1c4/0x364 [ 98.974719][ T426] kvm [426]: [<ffffffc009616928>] __kvm_nvhe_handle_trap+0xa8/0x130 [ 98.975152][ T426] kvm [426]: [<ffffffc009610064>] __kvm_nvhe___host_exit+0x64/0x64 [ 98.975588][ T426] ---- end of nVHE HYP call trace ----
---
Changes in v2:
- Fix printk warnings - %p expects (void *)
arch/arm64/kvm/handle_exit.c | 13 +++++-------- scripts/kallsyms.c | 2 +- 2 files changed, 6 insertions(+), 9 deletions(-)
arch/arm64/kvm/handle_exit.c +5 -8
@@ -296,13 +296,8 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr,u64 elr_in_kimg =__phys_to_kimg (elr_phys);u64 hyp_offset = elr_in_kimg -kaslr_offset () - elr_virt;u64 mode = spsr &PSR_MODE_MASK ; +u64 panic_addr = elr_virt + hyp_offset; -/* -* The nVHE hyp symbols are not included by kallsyms to avoid issues -* with aliasing. That means that the symbols cannot be printed with the -* "%pS" format specifier, so fall back to the vmlinux address if -* there's no better option. -*/ if (mode != PSR_MODE_EL2t && mode != PSR_MODE_EL2h) {kvm_err ("Invalid host exception to nVHE hyp!\n" ); } else if (ESR_ELx_EC(esr) == ESR_ELx_EC_BRK64 &&
@@ -322,9 +317,11 @@ void __noreturn __cold nvhe_hyp_panic_handler(u64 esr, u64 spsr,if (file)kvm_err ("nVHE hyp BUG at: %s:%u!\n" , file, line);else -kvm_err ("nVHE hyp BUG at: %016llx!\n" , elr_virt + hyp_offset); +kvm_err ("nVHE hyp BUG at: [<%016llx>] %pB!\n" , panic_addr, + (void *)panic_addr); }else { -kvm_err ("nVHE hyp panic at: %016llx!\n" , elr_virt + hyp_offset); +kvm_err ("nVHE hyp panic at: [<%016llx>] %pB!\n" , panic_addr, + (void *)panic_addr); }kvm_nvhe_dump_backtrace (hyp_offset);
scripts/kallsyms.c +1 -1
@@ -111,7 +111,7 @@ static bool is_ignored_symbol(const char *name, char type)".LASANPC" ,/* s390 kasan local symbols */ "__crc_" ,/* modversions */ "__efistub_" ,/* arm64 EFI stub namespace */ -"__kvm_nvhe_" ,/* arm64 non-VHE KVM namespace */ +"__kvm_nvhe_$" ,/* arm64 local symbols in non-VHE KVM namespace */ "__AArch64ADRPThunk_" ,/* arm64 lld */ "__ARMV5PILongThunk_" ,/* arm lld */ "__ARMV7PILongThunk_",
Signature · 1 line
2.35.1.574.g5d30c73bfb-goog