kvmppc: Paravirtualize KVM to support ultravisor
Subsystems (2)
v3 cover kvmppc: Paravirtualize KVM to support ultravisor
POWER platforms that supports the Protected Execution Facility (PEF) introduce features that combine hardware facilities and firmware to enable secure virtual machines. That includes a new processor mode (ultravisor mode) and the ultravisor firmware.
In PEF enabled systems, the ultravisor firmware runs at a privilege level above the hypervisor and also takes control over some system resources. The hypervisor, though, can make system calls to access these resources. Such system calls, a.k.a. ucalls, are handled by the ultravisor firmware.
The processor allows part of the system memory to be configured as secure memory, and introduces a a new mode, called secure mode, where any software entity in that mode can access secure memory. The hypervisor doesn't (and can't) run in secure mode, but a secure guest and the ultravisor firmware do.
This patch set adds support for ultravisor calls and do some preparation for running secure guests.
---
Changelog:
---v2->v3:
- Squashed patches:
"[PATCH v2 08/10] KVM: PPC: Ultravisor: Return to UV for hcalls from SVM" "[PATCH v2 09/10] KVM: PPC: Book3S HV: Fixed for running secure guests"
- Renamed patch from/to:
"[PATCH v2 08/10] KVM: PPC: Ultravisor: Return to UV for hcalls from SVM" "[PATCH v3 08/09] KVM: PPC: Ultravisor: Enter a secure guest
- Rebased
- Addressed comments from Paul Mackerras
- Dropped ultravisor checks made in power8 code
- Updated the commit message for:
"[PATCH v3 08/09] KVM: PPC: Ultravisor: Enter a secure guest"
- Addressed comments from Maddy
- Dropped imc-pmu.c changes
- Changed opal-imc.c to fail the probe when the ultravisor is enabled
- Fixed "ucall defined but not used" issue when CONFIG_PPC_UV not set
v1->v2:
- Addressed comments from Paul Mackerras:
- Write the pate in HV's table before doing that in UV's
- Renamed and better documented the ultravisor header files. Also added all possible return codes for each ucall
- Updated the commit message that introduces the MSR_S bit
- Moved ultravisor.c and ucall.S to arch/powerpc/kernel
- Changed ucall.S to not save CR
- Rebased
- Changed the patches order
- Updated several commit messages
- Added FW_FEATURE_ULTRAVISOR to enable use of firmware_has_feature()
- Renamed CONFIG_PPC_KVM_UV to CONFIG_PPC_UV and used it to ifdef the ucall handler and the code that populates the powerpc_firmware_features for ultravisor
- Exported the ucall symbol. KVM may be built as module.
- Restricted LDBAR access if the ultravisor firmware is available
- Dropped patches:
"[PATCH 06/13] KVM: PPC: Ultravisor: UV_RESTRICTED_SPR_WRITE ucall" "[PATCH 07/13] KVM: PPC: Ultravisor: UV_RESTRICTED_SPR_READ ucall" "[PATCH 08/13] KVM: PPC: Ultravisor: fix mtspr and mfspr"
- Squashed patches:
"[PATCH 09/13] KVM: PPC: Ultravisor: Return to UV for hcalls from SVM" "[PATCH 13/13] KVM: PPC: UV: Have fast_guest_return check secure_guest"
Anshuman Khandual (1):
KVM: PPC: Ultravisor: Add PPC_UV config option
Claudio Carvalho (2):
powerpc: Introduce FW_FEATURE_ULTRAVISOR
KVM: PPC: Ultravisor: Restrict LDBAR access
Michael Anderson (2):
KVM: PPC: Ultravisor: Use UV_WRITE_PATE ucall to register a PATE
KVM: PPC: Ultravisor: Check for MSR_S during hv_reset_msr
Ram Pai (2):
KVM: PPC: Ultravisor: Add generic ultravisor call handler
KVM: PPC: Ultravisor: Restrict flush of the partition tlb cache
Sukadev Bhattiprolu (2):
KVM: PPC: Ultravisor: Introduce the MSR_S bit
KVM: PPC: Ultravisor: Enter a secure guest
arch/powerpc/Kconfig | 20 +++++++ arch/powerpc/include/asm/firmware.h | 5 +- arch/powerpc/include/asm/kvm_host.h | 1 + arch/powerpc/include/asm/reg.h | 3 ++ arch/powerpc/include/asm/ultravisor-api.h | 24 +++++++++ arch/powerpc/include/asm/ultravisor.h | 49 +++++++++++++++++ arch/powerpc/kernel/Makefile | 1 + arch/powerpc/kernel/asm-offsets.c | 1 + arch/powerpc/kernel/prom.c | 6 +++ arch/powerpc/kernel/ucall.S | 31 +++++++++++ arch/powerpc/kernel/ultravisor.c | 28 ++++++++++ arch/powerpc/kvm/book3s_64_mmu_hv.c | 1 + arch/powerpc/kvm/book3s_hv_rmhandlers.S | 39 +++++++++++--- arch/powerpc/mm/book3s64/hash_utils.c | 3 +- arch/powerpc/mm/book3s64/pgtable.c | 65 +++++++++++++++++------ arch/powerpc/mm/book3s64/radix_pgtable.c | 9 ++-- arch/powerpc/platforms/powernv/idle.c | 6 ++- arch/powerpc/platforms/powernv/opal-imc.c | 7 +++ 18 files changed, 269 insertions(+), 30 deletions(-) create mode 100644 arch/powerpc/include/asm/ultravisor-api.h create mode 100644 arch/powerpc/include/asm/ultravisor.h create mode 100644 arch/powerpc/kernel/ucall.S create mode 100644 arch/powerpc/kernel/ultravisor.c
Signature · 1 line
2.20.1
PATCH v3 1/9 KVM: PPC: Ultravisor: Add PPC_UV config option
From: Anshuman Khandual <[email protected]>
CONFIG_PPC_UV adds support for ultravisor.
--- arch/powerpc/Kconfig | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
arch/powerpc/Kconfig +20 -0
@@ -439,6 +439,26 @@ config PPC_TRANSACTIONAL_MEM ---help--- Support user-mode Transactional Memory on POWERPC. +config PPC_UV + bool "Ultravisor support" + depends on KVM_BOOK3S_HV_POSSIBLE + select HMM_MIRROR + select HMM + select ZONE_DEVICE + select MIGRATE_VMA_HELPER + select DEV_PAGEMAP_OPS + select DEVICE_PRIVATE + select MEMORY_HOTPLUG + select MEMORY_HOTREMOVE + default n + help + This option paravirtualizes the kernel to run in POWER platforms that + supports the Protected Execution Facility (PEF). In such platforms, + the ultravisor firmware runs at a privilege level above the + hypervisor. + + If unsure, say "N". + config LD_HEAD_STUB_CATCH bool "Reserve 256 bytes to cope with linker stubs in HEAD text" if EXPERT depends on PPC64
Signature · 1 line
2.20.1
Re: [PATCH v3 1/9] KVM: PPC: Ultravisor: Add PPC_UV config option
Quoted 34 lines · Claudio Carvalho
From: Anshuman Khandual <[email protected]>
CONFIG_PPC_UVadds support for ultravisor.Signed-off-by Anshuman Khandual <[email protected]>Signed-off-by Bharata B Rao <[email protected]>Signed-off-by Ram Pai <[email protected]>[Update config help and commit message]Signed-off-by Claudio Carvalho <[email protected]>--- arch/powerpc/Kconfig | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)arch/powerpc/Kconfig +13
-0@@ -439,6 +439,26 @@ config PPC_TRANSACTIONAL_MEM ---help--- Support user-mode Transactional Memory on POWERPC. +config PPC_UV + bool "Ultravisor support" + depends on KVM_BOOK3S_HV_POSSIBLE + select HMM_MIRROR + select HMM + select ZONE_DEVICE + select MIGRATE_VMA_HELPER + select DEV_PAGEMAP_OPS + select DEVICE_PRIVATE + select MEMORY_HOTPLUG + select MEMORY_HOTREMOVE + default n + help
arch/powerpc/Kconfig config PPC_TRANSACTIONAL_MEM · line 464
Quoted 10 lines
arch/powerpc/Kconfig +7
-0+ This option paravirtualizes the kernel to run in POWER platforms that + supports the Protected Execution Facility (PEF). In such platforms, + the ultravisor firmware runs at a privilege level above the + hypervisor. + + If unsure, say "N". + config LD_HEAD_STUB_CATCH bool "Reserve 256 bytes to cope with linker stubs in HEAD text" if EXPERT depends on PPC64
Maybe this patch should be the last of the series, as it may cause some bisect trouble to have this option enabled while missing some of the patches.
Re: [PATCH v3 1/9] KVM: PPC: Ultravisor: Add PPC_UV config option
Quoted 41 lines · Leonardo Bras
Quoted 39 lines · Claudio Carvalho
From: Anshuman Khandual <[email protected]>
CONFIG_PPC_UVadds support for ultravisor.Signed-off-by Anshuman Khandual <[email protected]>Signed-off-by Bharata B Rao <[email protected]>Signed-off-by Ram Pai <[email protected]>[Update config help and commit message]Signed-off-by Claudio Carvalho <[email protected]>--- arch/powerpc/Kconfig | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)arch/powerpc/Kconfig +18
-0@@ -439,6 +439,26 @@ config PPC_TRANSACTIONAL_MEM ---help--- Support user-mode Transactional Memory on POWERPC. +config PPC_UV + bool "Ultravisor support" + depends on KVM_BOOK3S_HV_POSSIBLE + select HMM_MIRROR + select HMM + select ZONE_DEVICE + select MIGRATE_VMA_HELPER + select DEV_PAGEMAP_OPS + select DEVICE_PRIVATE + select MEMORY_HOTPLUG + select MEMORY_HOTREMOVE + default n + help + This option paravirtualizes the kernel to run in POWER platforms that + supports the Protected Execution Facility (PEF). In such platforms, + the ultravisor firmware runs at a privilege level above the + hypervisor. +
Quoted 8 lines
arch/powerpc/Kconfig config PPC_TRANSACTIONAL_MEM · line 464
Quoted 5 lines
arch/powerpc/Kconfig +2
-0+ If unsure, say "N". + config LD_HEAD_STUB_CATCH bool "Reserve 256 bytes to cope with linker stubs in HEAD text" if EXPERT depends on PPC64Maybe this patch should be the last of the series, as it may cause some bisect trouble to have this option enabled while missing some of the patches.
Thanks Leonardo. I changed that for the next version.
Claudio
PATCH v3 2/9 KVM: PPC: Ultravisor: Introduce the MSR_S bit
In reply to Claudio CarvalhoFrom: Sukadev Bhattiprolu <[email protected]>
The ultravisor processor mode is introduced in POWER platforms that supports the Protected Execution Facility (PEF). Ultravisor is higher privileged than hypervisor mode.
In PEF enabled platforms, the MSR_S bit is used to indicate if the thread is in secure state. With the MSR_S bit, the privilege state of the thread is now determined by MSR_S, MSR_HV and MSR_PR, as follows:
S HV PR
-----------------------
0 x 1 problem 1 0 1 problem x x 0 privileged x 1 0 hypervisor 1 1 0 ultravisor 1 1 1 reserved
The hypervisor doesn't (and can't) run with the MSR_S bit set, but a secure guest and the ultravisor firmware do.
--- arch/powerpc/include/asm/reg.h | 3 +++ 1 file changed, 3 insertions(+)
arch/powerpc/include/asm/reg.h +3 -0
@@ -38,6 +38,7 @@#define MSR_TM_LG 32/* Trans Mem Available */ #define MSR_VEC_LG 25/* Enable AltiVec */ #define MSR_VSX_LG 23/* Enable VSX */ +#define MSR_S_LG 22/* Secure VM bit */ #define MSR_POW_LG 18/* Enable Power Management */ #define MSR_WE_LG 18/* Wait State Enable */ #define MSR_TGPR_LG 17/* TLB Update registers in use */
@@ -71,11 +72,13 @@#define MSR_SF __MASK(MSR_SF_LG)/* Enable 64 bit mode */ #define MSR_ISF __MASK(MSR_ISF_LG)/* Interrupt 64b mode valid on 630 */ #define MSR_HV __MASK(MSR_HV_LG)/* Hypervisor state */ +#define MSR_S __MASK(MSR_S_LG)/* Secure state */ #else /* so tests for these bits fail on 32-bit */ #define MSR_SF 0#define MSR_ISF 0#define MSR_HV 0 +#define MSR_S 0#endif /*
Signature · 1 line
2.20.1
PATCH v3 3/9 powerpc: Introduce FW_FEATURE_ULTRAVISOR
In reply to Claudio CarvalhoThis feature tells if the ultravisor firmware is available to handle ucalls.
--- arch/powerpc/include/asm/firmware.h | 5 +++-- arch/powerpc/include/asm/ultravisor.h | 15 +++++++++++++++ arch/powerpc/kernel/Makefile | 1 + arch/powerpc/kernel/prom.c | 6 ++++++ arch/powerpc/kernel/ultravisor.c | 24 ++++++++++++++++++++++++ 5 files changed, 49 insertions(+), 2 deletions(-) create mode 100644 arch/powerpc/include/asm/ultravisor.h create mode 100644 arch/powerpc/kernel/ultravisor.c
arch/powerpc/include/asm/firmware.h +3 -2
@@ -54,6 +54,7 @@#define FW_FEATURE_DRC_INFO ASM_CONST(0x0000000800000000)#define FW_FEATURE_BLOCK_REMOVE ASM_CONST(0x0000001000000000)#define FW_FEATURE_PAPR_SCM ASM_CONST(0x0000002000000000) +#define FW_FEATURE_ULTRAVISOR ASM_CONST(0x0000004000000000)#ifndef __ASSEMBLY__
@@ -72,9 +73,9 @@ enum {FW_FEATURE_TYPE1_AFFINITY |FW_FEATURE_PRRN |FW_FEATURE_HPT_RESIZE |FW_FEATURE_DRMEM_V2 |FW_FEATURE_DRC_INFO |FW_FEATURE_BLOCK_REMOVE | -FW_FEATURE_PAPR_SCM , +FW_FEATURE_PAPR_SCM |FW_FEATURE_ULTRAVISOR ,FW_FEATURE_PSERIES_ALWAYS =0 , -FW_FEATURE_POWERNV_POSSIBLE =FW_FEATURE_OPAL , +FW_FEATURE_POWERNV_POSSIBLE =FW_FEATURE_OPAL |FW_FEATURE_ULTRAVISOR ,FW_FEATURE_POWERNV_ALWAYS =0 ,FW_FEATURE_PS3_POSSIBLE =FW_FEATURE_LPAR |FW_FEATURE_PS3_LV1 , FW_FEATURE_PS3_ALWAYS = FW_FEATURE_LPAR | FW_FEATURE_PS3_LV1,
arch/powerpc/include/asm/ultravisor.h new file +15 -0
@@ -0,0 +1,15 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Ultravisor definitions +* +* Copyright 2019, IBM Corporation. +* +*/ +#ifndef _ASM_POWERPC_ULTRAVISOR_H +#define _ASM_POWERPC_ULTRAVISOR_H + +/* Internal functions */ +extern int early_init_dt_scan_ultravisor (unsigned long node,const char *uname, +int depth,void *data); + +#endif /* _ASM_POWERPC_ULTRAVISOR_H */
arch/powerpc/kernel/Makefile +1 -0
@@ -154,6 +154,7 @@ endif obj-$(CONFIG_EPAPR_PARAVIRT) += epapr_paravirt.o epapr_hcalls.o obj-$(CONFIG_KVM_GUEST) += kvm.o kvm_emul.o +obj-$(CONFIG_PPC_UV) += ultravisor.o # Disable GCOV, KCOV & sanitizers in odd or sensitive code GCOV_PROFILE_prom_init.o := n
arch/powerpc/kernel/prom.c +6 -0
@@ -59,6 +59,7 @@#include <asm/firmware.h> #include <asm/dt_cpu_ftrs.h> #include <asm/drmem.h> +#include <asm/ultravisor.h> #include <mm/mmu_decl.h>
@@ -713,6 +714,11 @@ void __init early_init_devtree(void *params)of_scan_flat_dt (early_init_dt_scan_fw_dump,NULL );#endif +#if defined(CONFIG_PPC_UV ) +/* Scan tree for ultravisor feature */ +of_scan_flat_dt (early_init_dt_scan_ultravisor,NULL ); +#endif +/* Retrieve various informations from the /chosen node of the * device-tree, including the platform type, initrd location and * size, TCE reserve, and more ...
arch/powerpc/kernel/ultravisor.c new file +24 -0
@@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Ultravisor high level interfaces +* +* Copyright 2019, IBM Corporation. +* +*/ +#include <linux/init.h> +#include <linux/printk.h> +#include <linux/string.h> + +#include <asm/ultravisor.h> +#include <asm/firmware.h> + +int __init early_init_dt_scan_ultravisor (unsigned long node,const char *uname, +int depth,void *data) +{ +if (depth !=1 ||strcmp (uname,"ibm,ultravisor" ) !=0 ) +return 0 ; + + powerpc_firmware_features |=FW_FEATURE_ULTRAVISOR ; +pr_debug ("Ultravisor detected!\n" ); +return 1 ; +}
Signature · 1 line
2.20.1
Re: [PATCH v3 3/9] powerpc: Introduce FW_FEATURE_ULTRAVISOR
Quoted 2 lines · Claudio Carvalho
This feature tells if the ultravisor firmware is available to handle ucalls.
Everything in this patch that depends on CONFIG_PPC_UV should just depend on CONFIG_PPC_POWERNV instead. The reason is that every host kernel needs to be able to do the ultracall to set partition table entry 0, in case it ends up being run on a machine with an ultravisor. Otherwise we will have the situation where a host kernel may crash early in boot just because the machine it's booted on happens to have an ultravisor running. The crash will be a particularly nasty one because it will happen before we have probed the machine type and initialized the console; therefore it will just look like the machine hangs for no discernable reason.
We also need to think about how to provide a way for petitboot to know whether the kernel it is booting knows how to do a ucall to set its partition table entry. One suggestion would be to modify vmlinux.lds.S to add a new PT_NOTE entry in the program header of the binary with (say) a 64-bit doubleword which is a bitmap indicating capabilities of the binary. We would define the first bit as indicating that the kernel knows how to run under an ultravisor. When running under an ultravisor, petitboot could then look for the PT_NOTE and the ultravisor-capable bit in it, and if the PT_NOTE is not there or the bit is zero, put up a dialog warning the user that the kernel will probably crash early in boot, and asking for explicit confirmation that the user wants to proceed.
Paul.
Re: [PATCH v3 3/9] powerpc: Introduce FW_FEATURE_ULTRAVISOR
Quoted 14 lines · Paul Mackerras
Quoted 2 lines · Claudio Carvalho
This feature tells if the ultravisor firmware is available to handle ucalls.
Everything in this patch that depends on
CONFIG_PPC_UVshould just depend onCONFIG_PPC_POWERNVinstead. The reason is that every host kernel needs to be able to do the ultracall to set partition table entry 0, in case it ends up being run on a machine with an ultravisor. Otherwise we will have the situation where a host kernel may crash early in boot just because the machine it's booted on happens to have an ultravisor running. The crash will be a particularly nasty one because it will happen before we have probed the machine type and initialized the console; therefore it will just look like the machine hangs for no discernable reason.
Quoted 12 lines
We also need to think about how to provide a way for petitboot to know whether the kernel it is booting knows how to do a ucall to set its partition table entry. One suggestion would be to modify
vmlinux.lds.Sto add a newPT_NOTEentry in the program header of the binary with (say) a 64-bit doubleword which is a bitmap indicating capabilities of the binary. We would define the first bit as indicating that the kernel knows how to run under an ultravisor. When running under an ultravisor, petitboot could then look for thePT_NOTEand the ultravisor-capable bit in it, and if thePT_NOTEis not there or the bit is zero, put up a dialog warning the user that the kernel will probably crash early in boot, and asking for explicit confirmation that the user wants to proceed.
I just posted a separated RFC patch for the ELF note.
Thanks, Claudio.
Quoted 3 lines
Paul.
PATCH v3 4/9 KVM: PPC: Ultravisor: Add generic ultravisor call handler
In reply to Claudio CarvalhoFrom: Ram Pai <[email protected]>
Add the ucall() function, which can be used to make ultravisor calls with varied number of in and out arguments. Ultravisor calls can be made from the host or guests.
This copies the implementation of plpar_hcall().
[Change ucall.S to not save CR, rename and move the headers, build
ucall.S if CONFIG_PPC_UV set, and add some comments in the code]
--- arch/powerpc/include/asm/ultravisor-api.h | 20 +++++++++++++++ arch/powerpc/include/asm/ultravisor.h | 20 +++++++++++++++ arch/powerpc/kernel/Makefile | 2 +- arch/powerpc/kernel/ucall.S | 31 +++++++++++++++++++++++ arch/powerpc/kernel/ultravisor.c | 4 +++ 5 files changed, 76 insertions(+), 1 deletion(-) create mode 100644 arch/powerpc/include/asm/ultravisor-api.h create mode 100644 arch/powerpc/kernel/ucall.S
arch/powerpc/include/asm/ultravisor-api.h new file +20 -0
@@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Ultravisor calls. +* +* Copyright 2019, IBM Corporation. +* +*/ +#ifndef _ASM_POWERPC_ULTRAVISOR_API_H +#define _ASM_POWERPC_ULTRAVISOR_API_H + +#include <asm/hvcall.h> + +/* Return codes */ +#define U_NOT_AVAILABLE H_NOT_AVAILABLE +#define U_SUCCESS H_SUCCESS +#define U_FUNCTION H_FUNCTION +#define U_PARAMETER H_PARAMETER + +#endif /* _ASM_POWERPC_ULTRAVISOR_API_H */ +
arch/powerpc/include/asm/ultravisor.h +20 -0
@@ -8,8 +8,28 @@#ifndef _ASM_POWERPC_ULTRAVISOR_H#define _ASM_POWERPC_ULTRAVISOR_H +#include <asm/ultravisor-api.h> + +#if !defined(__ASSEMBLY__) +/* Internal functions */ extern int early_init_dt_scan_ultravisor (unsigned long node,const char *uname,int depth,void *data); +/* API functions */ +#define UCALL_BUFSIZE 4 +/** +* ucall: Make a powerpc ultravisor call. +* @opcode: The ultravisor call to make. +* @retbuf: Buffer to store up to 4 return arguments in. +* +* This call supports up to 6 arguments and 4 return arguments. Use +* UCALL_BUFSIZE to size the return argument buffer. +*/ +#if defined(CONFIG_PPC_UV ) +long ucall (unsigned long opcode,unsigned long *retbuf, ...); +#endif + +#endif /* !__ASSEMBLY__ */ +#endif /* _ASM_POWERPC_ULTRAVISOR_H */
arch/powerpc/kernel/Makefile +1 -1
@@ -154,7 +154,7 @@ endif obj-$(CONFIG_EPAPR_PARAVIRT) += epapr_paravirt.o epapr_hcalls.o obj-$(CONFIG_KVM_GUEST) += kvm.o kvm_emul.o -obj-$(CONFIG_PPC_UV) += ultravisor.o +obj-$(CONFIG_PPC_UV) += ultravisor.o ucall.o # Disable GCOV, KCOV & sanitizers in odd or sensitive code GCOV_PROFILE_prom_init.o := n
arch/powerpc/kernel/ucall.S new file +31 -0
@@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Generic code to perform an ultravisor call. + * + * Copyright 2019, IBM Corporation. + * + */ +#include <asm/ppc_asm.h> + +/* + * This function is based on the plpar_hcall() + */ +_GLOBAL_TOC(ucall) + mr r0,r3 + std r4,STK_PARAM(R4)(r1) /* Save ret buffer */ + mr r3,r5 + mr r4,r6 + mr r5,r7 + mr r6,r8 + mr r7,r9 + mr r8,r10 + + sc 2 /* invoke the ultravisor */ + + ld r12,STK_PARAM(R4)(r1) + std r4, 0(r12) + std r5, 8(r12) + std r6, 16(r12) + std r7, 24(r12) + + blr /* return r3 = status */
arch/powerpc/kernel/ultravisor.c +4 -0
@@ -8,10 +8,14 @@#include <linux/init.h> #include <linux/printk.h> #include <linux/string.h> +#include <linux/export.h> #include <asm/ultravisor.h> #include <asm/firmware.h> +/* in ucall.S */ +EXPORT_SYMBOL_GPL (ucall); +int __init early_init_dt_scan_ultravisor (unsigned long node,const char *uname,int depth,void *data) {
Signature · 1 line
2.20.1
Re: [PATCH v3 4/9] KVM: PPC: Ultravisor: Add generic ultravisor call handler
Quoted 7 lines · Claudio Carvalho
From: Ram Pai <[email protected]>
Add the
ucall()function, which can be used to make ultravisor calls with varied number of in and out arguments. Ultravisor calls can be made from the host or guests.This copies the implementation of
plpar_hcall().
Again, we will want all of this on every powernv-capable kernel, since they will all need to do UV_WRITE_PATE, even if they have no other support for the ultravisor.
Paul.
Re: [PATCH v3 4/9] KVM: PPC: Ultravisor: Add generic ultravisor call handler
In reply to Claudio CarvalhoQuoted 7 lines · Claudio Carvalho
From: Ram Pai <[email protected]>
Add the
ucall()function, which can be used to make ultravisor calls with varied number of in and out arguments. Ultravisor calls can be made from the host or guests.This copies the implementation of
plpar_hcall().
One point which I missed when I looked at this patch previously is that the ABI that we're defining here is different from the hcall ABI in that we are putting the ucall number in r0, whereas hcalls have the hcall number in r3. That makes ucalls more like syscalls, which have the syscall number in r0. So that last sentence quoted above is somewhat misleading.
The thing we need to consider is that when SMFCTRL[E] = 0, a ucall instruction becomes a hcall (that is, sc 2 is executed as if it was sc 1). In that case, the first argument to the ucall will be interpreted as the hcall number. Mostly that will happen not to be a valid hcall number, but sometimes it might unavoidably be a valid but unintended hcall number.
I think that will make it difficult to get ucalls to fail gracefully in the case where SMF/PEF is disabled. It seems like the assignment of ucall numbers was made so that they wouldn't overlap with valid hcall numbers; presumably that was so that we could tell when an hcall was actually intended to be a ucall. However, using a different GPR to pass the ucall number defeats that.
I realize that there is ultravisor code in development that takes the ucall number in r0, and also that having the ucall number in r3 would possibly make life more difficult for the place where we call UV_RETURN in assembler code. Nevertheless, perhaps we should consider changing the ABI to be like the hcall ABI before everything gets set in concrete.
Paul.
Re: Re: [PATCH v3 4/9] KVM: PPC: Ultravisor: Add generic ultravisor call handler
Quoted 23 lines · Paul Mackerras
Quoted 7 lines · Claudio Carvalho
From: Ram Pai <[email protected]>
Add the
ucall()function, which can be used to make ultravisor calls with varied number of in and out arguments. Ultravisor calls can be made from the host or guests.This copies the implementation of
plpar_hcall().One point which I missed when I looked at this patch previously is that the ABI that we're defining here is different from the hcall ABI in that we are putting the ucall number in r0, whereas hcalls have the hcall number in r3. That makes ucalls more like syscalls, which have the syscall number in r0. So that last sentence quoted above is somewhat misleading.
The thing we need to consider is that when SMFCTRL[E] = 0, a ucall instruction becomes a hcall (that is, sc 2 is executed as if it was sc 1). In that case, the first argument to the ucall will be interpreted as the hcall number. Mostly that will happen not to be a valid hcall number, but sometimes it might unavoidably be a valid but unintended hcall number.
Quoted 6 lines
I think that will make it difficult to get ucalls to fail gracefully in the case where SMF/PEF is disabled. It seems like the assignment of ucall numbers was made so that they wouldn't overlap with valid hcall numbers; presumably that was so that we could tell when an hcall was actually intended to be a ucall. However, using a different GPR to pass the ucall number defeats that.
Right this is a valid point. Glad that you caught it, otherwise it would have become a difficult to fix it in the future.
Quoted 5 lines
I realize that there is ultravisor code in development that takes the ucall number in r0, and also that having the ucall number in r3 would possibly make life more difficult for the place where we call
UV_RETURNin assembler code.
Its called from one place in the hypervisor, and the changes look simple.
- LOAD_REG_IMMEDIATE(r0, UV_RETURN)
+ LOAD_REG_IMMEDIATE(r3, UV_RETURN)
ld r7, VCPU_GPR(R7)(r4)
ld r6, VCPU_GPR(R6)(r4)
ld r4, VCPU_GPR(R4)(r4)What am i missing?
Quoted 3 lines
Nevertheless, perhaps we should consider
changing the ABI to be like the hcall ABI before everything gets set in concrete.
yes.
Thanks Paul!
RP
Re: [PATCH v3 4/9] KVM: PPC: Ultravisor: Add generic ultravisor call handler
In reply to Paul MackerrasHi Paul,
Quoted 6 lines · Paul Mackerras
The thing we need to consider is that when SMFCTRL[E] = 0, a ucall instruction becomes a hcall (that is, sc 2 is executed as if it was sc 1). In that case, the first argument to the ucall will be interpreted as the hcall number. Mostly that will happen not to be a valid hcall number, but sometimes it might unavoidably be a valid but unintended hcall number.
Shouldn't a caller of the ultravisor *know* that it is talking to the ultravisor in the first place? And not to the hypervisor.
Segher
Re: Re: [PATCH v3 4/9] KVM: PPC: Ultravisor: Add generic ultravisor call handler
Quoted 10 lines · Segher Boessenkool
Hi Paul,
Quoted 6 lines · Paul Mackerras
The thing we need to consider is that when SMFCTRL[E] = 0, a ucall instruction becomes a hcall (that is, sc 2 is executed as if it was sc 1). In that case, the first argument to the ucall will be interpreted as the hcall number. Mostly that will happen not to be a valid hcall number, but sometimes it might unavoidably be a valid but unintended hcall number.
Quoted 2 lines
Shouldn't a caller of the ultravisor *know* that it is talking to the ultravisor in the first place? And not to the hypervisor.
It may or may not. But if it knows and still decides to make the ucall,
the hypervisor must gracefully handle it.
We can't control who makes a ucall. A normal process within the VM could make a ucall too. Or a normal process running on top of the hypervisor could make a ucall.
RP
PATCH v3 5/9 KVM: PPC: Ultravisor: Use UV_WRITE_PATE ucall to register a PATE
In reply to Claudio CarvalhoFrom: Michael Anderson <[email protected]>
When running under an ultravisor, the ultravisor controls the real partition table and has it in secure memory where the hypervisor can't access it, and therefore we (the HV) have to do a ucall whenever we want to update an entry.
The HV still keeps a copy of its view of the partition table in normal memory so that the nest MMU can access it.
Both partition tables will have PATE entries for HV and normal virtual machines.
--- arch/powerpc/include/asm/ultravisor-api.h | 5 +++- arch/powerpc/include/asm/ultravisor.h | 14 ++++++++++ arch/powerpc/mm/book3s64/hash_utils.c | 3 +- arch/powerpc/mm/book3s64/pgtable.c | 34 +++++++++++++++++++++-- arch/powerpc/mm/book3s64/radix_pgtable.c | 9 ++++-- 5 files changed, 57 insertions(+), 8 deletions(-)
arch/powerpc/include/asm/ultravisor-api.h +4 -1
@@ -15,6 +15,9 @@#define U_SUCCESS H_SUCCESS#define U_FUNCTION H_FUNCTION#define U_PARAMETER H_PARAMETER +#define U_PERMISSION H_PERMISSION -#endif /* _ASM_POWERPC_ULTRAVISOR_API_H */ +/* opcodes */ +#define UV_WRITE_PATE 0xF104 +#endif /* _ASM_POWERPC_ULTRAVISOR_API_H */
arch/powerpc/include/asm/ultravisor.h +14 -0
@@ -12,6 +12,8 @@#if !defined(__ASSEMBLY__) +#include <linux/types.h> +/* Internal functions */ extern int early_init_dt_scan_ultravisor (unsigned long node,const char *uname,int depth,void *data);
@@ -28,8 +30,20 @@ extern int early_init_dt_scan_ultravisor(unsigned long node, const char *uname,*/ #if defined(CONFIG_PPC_UV )long ucall (unsigned long opcode,unsigned long *retbuf, ...); +#else +static long ucall (unsigned long opcode,unsigned long *retbuf, ...) +{ +return U_NOT_AVAILABLE ; +}#endif +static inline int uv_register_pate (u64 lpid,u64 dw0,u64 dw1) +{ +unsigned long retbuf[UCALL_BUFSIZE ]; + +return ucall (UV_WRITE_PATE , retbuf, lpid, dw0, dw1); +} +#endif /* !__ASSEMBLY__ */ #endif /* _ASM_POWERPC_ULTRAVISOR_H */
arch/powerpc/mm/book3s64/hash_utils.c +2 -1
@@ -1080,9 +1080,10 @@ void hash__early_init_mmu_secondary(void)if (!cpu_has_feature (CPU_FTR_ARCH_300 ))mtspr (SPRN_SDR1 , _SDR1); -else +else if (!firmware_has_feature (FW_FEATURE_ULTRAVISOR ))mtspr (SPRN_PTCR ,__pa (partition_tb) | (PATB_SIZE_SHIFT -12 )); + }/* Initialize SLB */ slb_initialize ();
arch/powerpc/mm/book3s64/pgtable.c +31 -3
@@ -16,6 +16,8 @@#include <asm/tlb.h> #include <asm/trace.h> #include <asm/powernv.h> +#include <asm/firmware.h> +#include <asm/ultravisor.h> #include <mm/mmu_decl.h> #include <trace/events/thp.h>
@@ -206,12 +208,25 @@ void __init mmu_partition_table_init(void)* 64 K size. */ ptcr =__pa (partition_tb) | (PATB_SIZE_SHIFT -12 ); -mtspr (SPRN_PTCR , ptcr); +/* +* If ultravisor is available, it is responsible for creating and +* managing partition table +*/ +if (!firmware_has_feature (FW_FEATURE_ULTRAVISOR )) +mtspr (SPRN_PTCR , ptcr); + +/* +* Since nestMMU cannot access secure memory. Create +* and manage our own partition table. This table +* contains entries for nonsecure and hypervisor +* partition. +*/ powernv_set_nmmu_ptcr (ptcr); } -void mmu_partition_table_set_entry (unsigned int lpid,unsigned long dw0, -unsigned long dw1) +static void __mmu_partition_table_set_entry (unsigned int lpid, +unsigned long dw0, +unsigned long dw1) {unsigned long old =be64_to_cpu (partition_tb[lpid].patb0);
@@ -238,6 +253,19 @@ void mmu_partition_table_set_entry(unsigned int lpid, unsigned long dw0,/* do we need fixup here ?*/ asmvolatile ("eieio; tlbsync; ptesync" : : :"memory" ); } + +void mmu_partition_table_set_entry (unsigned int lpid,unsigned long dw0, +unsigned long dw1) +{ +__mmu_partition_table_set_entry (lpid, dw0, dw1); + +if (firmware_has_feature (FW_FEATURE_ULTRAVISOR )) { +uv_register_pate (lpid, dw0, dw1); +pr_info ("PATE registered by ultravisor: dw0 = 0x%lx, dw1 = 0x%lx\n" , + dw0, dw1); + } +} +EXPORT_SYMBOL_GPL (mmu_partition_table_set_entry);static pmd_t *get_pmd_from_cache (struct mm_struct *mm)
arch/powerpc/mm/book3s64/radix_pgtable.c +6 -3
@@ -655,8 +655,10 @@ void radix__early_init_mmu_secondary(void) lpcr =mfspr (SPRN_LPCR );mtspr (SPRN_LPCR , lpcr |LPCR_UPRT |LPCR_HR ); -mtspr (SPRN_PTCR , -__pa (partition_tb) | (PATB_SIZE_SHIFT -12 )); +if (!firmware_has_feature (FW_FEATURE_ULTRAVISOR )) +mtspr (SPRN_PTCR ,__pa (partition_tb) | + (PATB_SIZE_SHIFT -12 )); +radix_init_amor (); }
@@ -672,7 +674,8 @@ void radix__mmu_cleanup_all(void)if (!firmware_has_feature (FW_FEATURE_LPAR )) { lpcr =mfspr (SPRN_LPCR );mtspr (SPRN_LPCR , lpcr & ~LPCR_UPRT ); -mtspr (SPRN_PTCR ,0 ); +if (!firmware_has_feature (FW_FEATURE_ULTRAVISOR )) +mtspr (SPRN_PTCR ,0 );powernv_set_nmmu_ptcr (0 );radix__flush_tlb_all (); }
Signature · 1 line
2.20.1
Re: [PATCH v3 5/9] KVM: PPC: Ultravisor: Use UV_WRITE_PATE ucall to register a PATE
Quoted 7 lines · Claudio Carvalho
From: Michael Anderson <[email protected]>
When running under an ultravisor, the ultravisor controls the real partition table and has it in secure memory where the hypervisor can't access it, and therefore we (the HV) have to do a ucall whenever we want to update an entry.
Quoted 5 lines
The HV still keeps a copy of its view of the partition table in normal memory so that the nest MMU can access it.
Both partition tables will have PATE entries for HV and normal virtual machines.
As discussed before, all of this should depend only on CONFIG_PPC_POWERNV.
Paul.
PATCH v3 6/9 KVM: PPC: Ultravisor: Restrict flush of the partition tlb cache
In reply to Claudio CarvalhoFrom: Ram Pai <[email protected]>
Ultravisor is responsible for flushing the tlb cache, since it manages the PATE entries. Hence skip tlb flush, if the ultravisor firmware is available.
--- arch/powerpc/mm/book3s64/pgtable.c | 33 +++++++++++++++++------------- 1 file changed, 19 insertions(+), 14 deletions(-)
arch/powerpc/mm/book3s64/pgtable.c +19 -14
@@ -224,6 +224,23 @@ void __init mmu_partition_table_init(void)powernv_set_nmmu_ptcr (ptcr); } +static void flush_partition (unsigned int lpid,unsigned long dw0) +{ +if (dw0 &PATB_HR ) { + asm volatile(PPC_TLBIE_5(%0, %1, 2, 0, 1) : : + "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid)); + asm volatile(PPC_TLBIE_5(%0, %1, 2, 1, 1) : : + "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid)); +trace_tlbie (lpid,0 ,TLBIEL_INVAL_SET_LPID , lpid,2 ,0 ,1 ); + }else { + asm volatile(PPC_TLBIE_5(%0, %1, 2, 0, 0) : : + "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid)); +trace_tlbie (lpid,0 ,TLBIEL_INVAL_SET_LPID , lpid,2 ,0 ,0 ); + } +/* do we need fixup here ?*/ + asmvolatile ("eieio; tlbsync; ptesync" : : :"memory" ); +} +static void __mmu_partition_table_set_entry (unsigned int lpid,unsigned long dw0,unsigned long dw1)
@@ -238,20 +255,8 @@ static void __mmu_partition_table_set_entry(unsigned int lpid,* The type of flush (hash or radix) depends on what the previous * use of this partition ID was, not the new use. */ - asmvolatile ("ptesync" : : :"memory" ); -if (old &PATB_HR ) { - asm volatile(PPC_TLBIE_5(%0,%1,2,0,1) : : - "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid)); - asm volatile(PPC_TLBIE_5(%0,%1,2,1,1) : : - "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid)); -trace_tlbie (lpid,0 ,TLBIEL_INVAL_SET_LPID , lpid,2 ,0 ,1 ); - }else { - asm volatile(PPC_TLBIE_5(%0,%1,2,0,0) : : - "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid)); -trace_tlbie (lpid,0 ,TLBIEL_INVAL_SET_LPID , lpid,2 ,0 ,0 ); - } -/* do we need fixup here ?*/ - asmvolatile ("eieio; tlbsync; ptesync" : : :"memory" ); +if (!firmware_has_feature (FW_FEATURE_ULTRAVISOR )) +flush_partition (lpid, old); } void mmu_partition_table_set_entry(unsigned int lpid, unsigned long dw0,
Signature · 1 line
2.20.1
Re: [PATCH v3 6/9] KVM: PPC: Ultravisor: Restrict flush of the partition tlb cache
Quoted 27 lines · Claudio Carvalho
From: Ram Pai <[email protected]>
Ultravisor is responsible for flushing the tlb cache, since it manages the PATE entries. Hence skip tlb flush, if the ultravisor firmware is available.
Signed-off-by Ram Pai <[email protected]>Signed-off-by Claudio Carvalho <[email protected]>--- arch/powerpc/mm/book3s64/pgtable.c | 33 +++++++++++++++++------------- 1 file changed, 19 insertions(+), 14 deletions(-)arch/powerpc/mm/book3s64/pgtable.c +7
-0@@ -224,6 +224,23 @@ void __init mmu_partition_table_init(void)powernv_set_nmmu_ptcr (ptcr); } +static void flush_partition (unsigned int lpid,unsigned long dw0) +{ +if (dw0 &PATB_HR ) { + asm volatile(PPC_TLBIE_5(%0, %1, 2, 0, 1) : : + "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid)); + asm volatile(PPC_TLBIE_5(%0, %1, 2, 1, 1) : : + "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid));
arch/powerpc/mm/book3s64/pgtable.c void __init mmu_partition_table_init(void) · line 243
Quoted 10 lines
arch/powerpc/mm/book3s64/pgtable.c +10
-0+trace_tlbie (lpid,0 ,TLBIEL_INVAL_SET_LPID , lpid,2 ,0 ,1 ); + }else { + asm volatile(PPC_TLBIE_5(%0, %1, 2, 0, 0) : : + "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid)); +trace_tlbie (lpid,0 ,TLBIEL_INVAL_SET_LPID , lpid,2 ,0 ,0 ); + } +/* do we need fixup here ?*/ + asmvolatile ("eieio; tlbsync; ptesync" : : :"memory" ); +} +
checkpatch.pl seems to complain:
ERROR: need consistent spacing around '%' (ctx:WxV) #125: FILE: arch/powerpc/mm/book3s64/pgtable.c:230:
+ asm volatile(PPC_TLBIE_5(%0, %1, 2, 0, 1) : :
^
ERROR: need consistent spacing around '%' (ctx:WxV) #127: FILE: arch/powerpc/mm/book3s64/pgtable.c:232:
+ asm volatile(PPC_TLBIE_5(%0, %1, 2, 1, 1) : :
^
ERROR: need consistent spacing around '%' (ctx:WxV) #131: FILE: arch/powerpc/mm/book3s64/pgtable.c:236:
+ asm volatile(PPC_TLBIE_5(%0, %1, 2, 0, 0) : :
^
Quoted 28 lines
arch/powerpc/mm/book3s64/pgtable.c +2
-15static void __mmu_partition_table_set_entry (unsigned int lpid,unsigned long dw0,unsigned long dw1)@@ -238,20 +255,8 @@ static void __mmu_partition_table_set_entry(unsigned int lpid,* The type of flush (hash or radix) depends on what the previous * use of this partition ID was, not the new use. */ - asmvolatile ("ptesync" : : :"memory" ); -if (old &PATB_HR ) { - asm volatile(PPC_TLBIE_5(%0,%1,2,0,1) : : - "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid)); - asm volatile(PPC_TLBIE_5(%0,%1,2,1,1) : : - "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid)); -trace_tlbie (lpid,0 ,TLBIEL_INVAL_SET_LPID , lpid,2 ,0 ,1 ); - }else { - asm volatile(PPC_TLBIE_5(%0,%1,2,0,0) : : - "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid)); -trace_tlbie (lpid,0 ,TLBIEL_INVAL_SET_LPID , lpid,2 ,0 ,0 ); - } -/* do we need fixup here ?*/ - asmvolatile ("eieio; tlbsync; ptesync" : : :"memory" ); +if (!firmware_has_feature (FW_FEATURE_ULTRAVISOR )) +flush_partition (lpid, old); } void mmu_partition_table_set_entry(unsigned int lpid, unsigned long dw0, --2.20.1
Re: [PATCH v3 6/9] KVM: PPC: Ultravisor: Restrict flush of the partition tlb cache
Quoted 50 lines · Murilo Opsfelder Araújo
Quoted 27 lines · Claudio Carvalho
From: Ram Pai <[email protected]>
Ultravisor is responsible for flushing the tlb cache, since it manages the PATE entries. Hence skip tlb flush, if the ultravisor firmware is available.
Signed-off-by Ram Pai <[email protected]>Signed-off-by Claudio Carvalho <[email protected]>--- arch/powerpc/mm/book3s64/pgtable.c | 33 +++++++++++++++++------------- 1 file changed, 19 insertions(+), 14 deletions(-)arch/powerpc/mm/book3s64/pgtable.c +7
-0@@ -224,6 +224,23 @@ void __init mmu_partition_table_init(void)powernv_set_nmmu_ptcr (ptcr); } +static void flush_partition (unsigned int lpid,unsigned long dw0) +{ +if (dw0 &PATB_HR ) { + asm volatile(PPC_TLBIE_5(%0, %1, 2, 0, 1) : : + "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid)); + asm volatile(PPC_TLBIE_5(%0, %1, 2, 1, 1) : : + "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid));arch/powerpc/mm/book3s64/pgtable.c void __init mmu_partition_table_init(void) · line 243
Quoted 10 lines
arch/powerpc/mm/book3s64/pgtable.c +10
-0+trace_tlbie (lpid,0 ,TLBIEL_INVAL_SET_LPID , lpid,2 ,0 ,1 ); + }else { + asm volatile(PPC_TLBIE_5(%0, %1, 2, 0, 0) : : + "r" (TLBIEL_INVAL_SET_LPID), "r" (lpid)); +trace_tlbie (lpid,0 ,TLBIEL_INVAL_SET_LPID , lpid,2 ,0 ,0 ); + } +/* do we need fixup here ?*/ + asmvolatile ("eieio; tlbsync; ptesync" : : :"memory" ); +} +checkpatch.pl seems to complain:
ERROR: need consistent spacing around '%' (ctx:WxV) #125: FILE:
arch/powerpc/mm/book3s64/pgtable.c:230:
+ asm volatile(PPC_TLBIE_5(%0, %1, 2, 0, 1) : :^ERROR: need consistent spacing around '%' (ctx:WxV) #127: FILE:
arch/powerpc/mm/book3s64/pgtable.c:232:
+ asm volatile(PPC_TLBIE_5(%0, %1, 2, 1, 1) : :
Quoted 6 lines
^ERROR: need consistent spacing around '%' (ctx:WxV) #131: FILE:
arch/powerpc/mm/book3s64/pgtable.c:236:
+ asm volatile(PPC_TLBIE_5(%0, %1, 2, 0, 0) : :^
Then clearly checkpatch.pl has a bug.
Paul.
PATCH v3 7/9 KVM: PPC: Ultravisor: Restrict LDBAR access
In reply to Claudio CarvalhoWhen the ultravisor firmware is available, it takes control over the LDBAR register. In this case, thread-imc updates and save/restore operations on the LDBAR register are handled by ultravisor.
--- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 2 ++ arch/powerpc/platforms/powernv/idle.c | 6 ++++-- arch/powerpc/platforms/powernv/opal-imc.c | 7 +++++++ 3 files changed, 13 insertions(+), 2 deletions(-)
arch/powerpc/kvm/book3s_hv_rmhandlers.S +2 -0
@@ -375,8 +375,10 @@ BEGIN_FTR_SECTION mtspr SPRN_RPR, r0 ld r0, KVM_SPLIT_PMMAR(r6) mtspr SPRN_PMMAR, r0 +BEGIN_FW_FTR_SECTION_NESTED(70) ld r0, KVM_SPLIT_LDBAR(r6) mtspr SPRN_LDBAR, r0 +END_FW_FTR_SECTION_NESTED(FW_FEATURE_ULTRAVISOR, 0, 70) isync FTR_SECTION_ELSE /* On P9 we use the split_info for coordinating LPCR changes */
arch/powerpc/platforms/powernv/idle.c +4 -2
@@ -679,7 +679,8 @@ static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on) sprs.ptcr =mfspr (SPRN_PTCR ); sprs.rpr =mfspr (SPRN_RPR ); sprs.tscr =mfspr (SPRN_TSCR ); - sprs.ldbar =mfspr (SPRN_LDBAR ); +if (!firmware_has_feature (FW_FEATURE_ULTRAVISOR )) + sprs.ldbar =mfspr (SPRN_LDBAR ); sprs_saved = true;
@@ -762,7 +763,8 @@ static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on)mtspr (SPRN_PTCR , sprs.ptcr);mtspr (SPRN_RPR , sprs.rpr);mtspr (SPRN_TSCR , sprs.tscr); -mtspr (SPRN_LDBAR , sprs.ldbar); +if (!firmware_has_feature (FW_FEATURE_ULTRAVISOR )) +mtspr (SPRN_LDBAR , sprs.ldbar);if (pls >= pnv_first_tb_loss_level) {/* TB loss */
arch/powerpc/platforms/powernv/opal-imc.c +7 -0
@@ -254,6 +254,13 @@ static int opal_imc_counters_probe(struct platform_device *pdev)bool core_imc_reg = false, thread_imc_reg = false;u32 type; +/* +* When the Ultravisor is enabled, it is responsible for thread-imc +* updates +*/ +if (firmware_has_feature (FW_FEATURE_ULTRAVISOR )) +return -EACCES ; +/* * Check whether this is kdump kernel. If yes, force the engines to * stop and return.
Signature · 1 line
2.20.1
Re: [PATCH v3 7/9] KVM: PPC: Ultravisor: Restrict LDBAR access
Quoted 56 lines · Claudio Carvalho
When the ultravisor firmware is available, it takes control over the LDBAR register. In this case, thread-imc updates and save/restore operations on the LDBAR register are handled by ultravisor.
Signed-off-by Claudio Carvalho <[email protected]>Signed-off-by Ram Pai <[email protected]>--- arch/powerpc/kvm/book3s_hv_rmhandlers.S | 2 ++ arch/powerpc/platforms/powernv/idle.c | 6 ++++-- arch/powerpc/platforms/powernv/opal-imc.c | 7 +++++++ 3 files changed, 13 insertions(+), 2 deletions(-)arch/powerpc/kvm/book3s_hv_rmhandlers.S +2
-0@@ -375,8 +375,10 @@ BEGIN_FTR_SECTION mtspr SPRN_RPR, r0 ld r0, KVM_SPLIT_PMMAR(r6) mtspr SPRN_PMMAR, r0 +BEGIN_FW_FTR_SECTION_NESTED(70) ld r0, KVM_SPLIT_LDBAR(r6) mtspr SPRN_LDBAR, r0 +END_FW_FTR_SECTION_NESTED(FW_FEATURE_ULTRAVISOR, 0, 70) isync FTR_SECTION_ELSE /* On P9 we use the split_info for coordinating LPCR changes */arch/powerpc/platforms/powernv/idle.c +4
-2@@ -679,7 +679,8 @@ static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on) sprs.ptcr =mfspr (SPRN_PTCR ); sprs.rpr =mfspr (SPRN_RPR ); sprs.tscr =mfspr (SPRN_TSCR ); - sprs.ldbar =mfspr (SPRN_LDBAR ); +if (!firmware_has_feature (FW_FEATURE_ULTRAVISOR )) + sprs.ldbar =mfspr (SPRN_LDBAR ); sprs_saved = true;@@ -762,7 +763,8 @@ static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on)mtspr (SPRN_PTCR , sprs.ptcr);mtspr (SPRN_RPR , sprs.rpr);mtspr (SPRN_TSCR , sprs.tscr); -mtspr (SPRN_LDBAR , sprs.ldbar); +if (!firmware_has_feature (FW_FEATURE_ULTRAVISOR )) +mtspr (SPRN_LDBAR , sprs.ldbar);if (pls >= pnv_first_tb_loss_level) {/* TB loss */ arch/powerpc/platforms/powernv/opal-imc.c +0
-0
arch/powerpc/platforms/powernv/opal-imc.c static int opal_imc_counters_probe(struct platform_device *pdev) · line 260
Quoted 7 lines
arch/powerpc/platforms/powernv/opal-imc.c +4
-0@@ -254,6 +254,13 @@ static int opal_imc_counters_probe(struct platform_device *pdev)bool core_imc_reg = false, thread_imc_reg = false;u32 type; +/* +* When the Ultravisor is enabled, it is responsible for thread-imc +* updates +*/
Would prefer the comment to be "Disable IMC devices, when Ultravisor is enabled"
Rest looks good.
Quoted 6 lines
arch/powerpc/platforms/powernv/opal-imc.c +3
-0+if (firmware_has_feature (FW_FEATURE_ULTRAVISOR )) +return -EACCES ; +/* * Check whether this is kdump kernel. If yes, force the engines to * stop and return.
Re: [PATCH v3 7/9] KVM: PPC: Ultravisor: Restrict LDBAR access
Quoted 75 lines · Madhavan Srinivasan
Quoted 61 lines · Claudio Carvalho
When the ultravisor firmware is available, it takes control over the LDBAR register. In this case, thread-imc updates and save/restore operations on the LDBAR register are handled by ultravisor.
Signed-off-by Claudio Carvalho <[email protected]>Signed-off-by Ram Pai <[email protected]>---
arch/powerpc/kvm/book3s_hv_rmhandlers.S | 2 ++ arch/powerpc/platforms/powernv/idle.c | 6 ++++--arch/powerpc/platforms/powernv/opal-imc.c| 7 +++++++3 files changed, 13 insertions(+), 2 deletions(-)arch/powerpc/kvm/book3s_hv_rmhandlers.S +0
-0@@ -375,8 +375,10 @@ BEGIN_FTR_SECTIONmtspr
SPRN_RPR, r0
ld r0,KVM_SPLIT_PMMAR(r6)
mtsprSPRN_PMMAR, r0 +BEGIN_FW_FTR_SECTION_NESTED(70)
ld r0,KVM_SPLIT_LDBAR(r6)
mtsprSPRN_LDBAR, r0 +END_FW_FTR_SECTION_NESTED(FW_FEATURE_ULTRAVISOR, 0, 70) isync
FTR_SECTION_ELSE
/* On P9 we use thesplit_infofor coordinating LPCR changes */arch/powerpc/platforms/powernv/idle.c +4
-2@@ -679,7 +679,8 @@ static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on) sprs.ptcr = mfspr(SPRN_PTCR); sprs.rpr = mfspr(SPRN_RPR); sprs.tscr = mfspr(SPRN_TSCR); - sprs.ldbar = mfspr(SPRN_LDBAR); + if (!firmware_has_feature(FW_FEATURE_ULTRAVISOR)) + sprs.ldbar = mfspr(SPRN_LDBAR); sprs_saved = true;@@ -762,7 +763,8 @@ static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on) mtspr(SPRN_PTCR, sprs.ptcr); mtspr(SPRN_RPR, sprs.rpr); mtspr(SPRN_TSCR, sprs.tscr); - mtspr(SPRN_LDBAR, sprs.ldbar); + if (!firmware_has_feature(FW_FEATURE_ULTRAVISOR)) + mtspr(SPRN_LDBAR, sprs.ldbar); if (pls >= pnv_first_tb_loss_level) {/* TB loss */
arch/powerpc/platforms/powernv/opal-imc.c +0
-0arch/powerpc/platforms/powernv/opal-imc.c static int opal_imc_counters_probe(struct · line 261
Quoted 8 lines
arch/powerpc/platforms/powernv/opal-imc.c +4
-0@@ -254,6 +254,13 @@ static int opal_imc_counters_probe(struct platform_device *pdev) bool core_imc_reg = false, thread_imc_reg = false; u32 type; + /* +* When the Ultravisor is enabled, it is responsible for thread-imc +* updates +*/ Would prefer the comment to be "Disable IMC devices, when Ultravisor is enabled"
Rest looks good.
Quoted 1 line
Thanks Maddy. I applied that change to the next version.
Claudio
Quoted 7 lines
Quoted 7 lines
arch/powerpc/platforms/powernv/opal-imc.c +3
-0+ if (firmware_has_feature(FW_FEATURE_ULTRAVISOR)) + return -EACCES; + /** Check whether this is kdump kernel. If yes, force the engines to * stop and return.
Re: [PATCH v3 7/9] KVM: PPC: Ultravisor: Restrict LDBAR access
In reply to Claudio CarvalhoQuoted 6 lines · Claudio Carvalho
When the ultravisor firmware is available, it takes control over the LDBAR register. In this case, thread-imc updates and save/restore operations on the LDBAR register are handled by ultravisor.
Signed-off-by Claudio Carvalho <[email protected]>Signed-off-by Ram Pai <[email protected]>
Just a note on the signed-off-by: it's a bit weird to have Ram's signoff when he is neither the author nor the sender of the patch. The author is assumed to be Claudio; if that is not correct then the patch should have a From: line at the start telling us who the author is, and ideally that person should have a signoff line before Claudio's signoff as the sender of the patch.
Paul.
Re: Re: [PATCH v3 7/9] KVM: PPC: Ultravisor: Restrict LDBAR access
Quoted 8 lines · Paul Mackerras
Quoted 6 lines · Claudio Carvalho
When the ultravisor firmware is available, it takes control over the LDBAR register. In this case, thread-imc updates and save/restore operations on the LDBAR register are handled by ultravisor.
Signed-off-by Claudio Carvalho <[email protected]>Signed-off-by Ram Pai <[email protected]>
Quoted 8 lines
Acked-by Paul Mackerras <[email protected]>Just a note on the signed-off-by: it's a bit weird to have Ram's signoff when he is neither the author nor the sender of the patch. The author is assumed to be Claudio; if that is not correct then the patch should have a From: line at the start telling us who the author is, and ideally that person should have a signoff line before Claudio's signoff as the sender of the patch.
Ryan originally wrote this patch, which I than modified, before Claudio further modified it to its current form.
So I think the author should be Ryan.
RP
PATCH v3 8/9 KVM: PPC: Ultravisor: Enter a secure guest
In reply to Claudio CarvalhoFrom: Sukadev Bhattiprolu <[email protected]>
To enter a secure guest, we have to go through the ultravisor, therefore we do a ucall when we are entering a secure guest.
This change is needed for any sort of entry to the secure guest from the hypervisor, whether it is a return from an hcall, a return from a hypervisor interrupt, or the first time that a secure guest vCPU is run.
If we are returning from an hcall, the results are already in the appropriate registers (R3:12), except for R6,7, which need to be restored before doing the ucall (UV_RETURN).
Have fast_guest_return check the kvm_arch.secure_guest field so that a new CPU enters UV when started (in response to a RTAS start-cpu call).
Thanks to input from Paul Mackerras, Ram Pai and Mike Anderson.
[Pass SRR1 in r11 for UV_RETURN, fix kvmppc_msr_interrupt to preserve
the MSR_S bit]
--- arch/powerpc/include/asm/kvm_host.h | 1 + arch/powerpc/include/asm/ultravisor-api.h | 1 + arch/powerpc/kernel/asm-offsets.c | 1 + arch/powerpc/kvm/book3s_hv_rmhandlers.S | 37 +++++++++++++++++++---- 4 files changed, 34 insertions(+), 6 deletions(-)
arch/powerpc/include/asm/kvm_host.h +1 -0
@@ -294,6 +294,7 @@ struct kvm_arch {cpumask_t cpu_in_guest;u8 radix;u8 fwnmi_enabled; +u8 secure_guest;bool threads_indep;bool nested_enable;pgd_t *pgtable;
arch/powerpc/include/asm/ultravisor-api.h +1 -0
@@ -19,5 +19,6 @@/* opcodes */ #define UV_WRITE_PATE 0xF104 +#define UV_RETURN 0xF11C#endif /* _ASM_POWERPC_ULTRAVISOR_API_H */
arch/powerpc/kernel/asm-offsets.c +1 -0
@@ -508,6 +508,7 @@ int main(void)OFFSET (KVM_VRMA_SLB_V , kvm, arch.vrma_slb_v);OFFSET (KVM_RADIX , kvm, arch.radix);OFFSET (KVM_FWNMI , kvm, arch.fwnmi_enabled); +OFFSET (KVM_SECURE_GUEST , kvm, arch.secure_guest);OFFSET (VCPU_DSISR , kvm_vcpu, arch.shregs.dsisr);OFFSET (VCPU_DAR , kvm_vcpu, arch.shregs.dar);OFFSET (VCPU_VPA , kvm_vcpu, arch.vpa.pinned_addr);
arch/powerpc/kvm/book3s_hv_rmhandlers.S +31 -6
@@ -36,6 +36,7 @@ #include <asm/asm-compat.h> #include <asm/feature-fixups.h> #include <asm/cpuidle.h> +#include <asm/ultravisor-api.h> /* Sign-extend HDEC if not on POWER9 */ #define EXTEND_HDEC(reg) \
@@ -1092,16 +1093,12 @@ BEGIN_FTR_SECTION END_FTR_SECTION_IFSET(CPU_FTR_HAS_PPR) ld r5, VCPU_LR(r4) - ld r6, VCPU_CR(r4) mtlr r5 - mtcr r6 ld r1, VCPU_GPR(R1)(r4) ld r2, VCPU_GPR(R2)(r4) ld r3, VCPU_GPR(R3)(r4) ld r5, VCPU_GPR(R5)(r4) - ld r6, VCPU_GPR(R6)(r4) - ld r7, VCPU_GPR(R7)(r4) ld r8, VCPU_GPR(R8)(r4) ld r9, VCPU_GPR(R9)(r4) ld r10, VCPU_GPR(R10)(r4)
@@ -1119,10 +1116,35 @@ BEGIN_FTR_SECTION mtspr SPRN_HDSISR, r0 END_FTR_SECTION_IFSET(CPU_FTR_ARCH_300) + ld r6, VCPU_KVM(r4) + lbz r7, KVM_SECURE_GUEST(r6) + cmpdi r7, 0 + bne ret_to_ultra + + lwz r6, VCPU_CR(r4) + mtcr r6 + + ld r7, VCPU_GPR(R7)(r4) + ld r6, VCPU_GPR(R6)(r4) ld r0, VCPU_GPR(R0)(r4) ld r4, VCPU_GPR(R4)(r4) HRFI_TO_GUEST b . +/* + * We are entering a secure guest, so we have to invoke the ultravisor to do + * that. If we are returning from a hcall, the results are already in the + * appropriate registers (R3:12), except for R6,7 which we used as temporary + * registers above. Restore them, and set R0 to the ucall number (UV_RETURN). + */ +ret_to_ultra: + lwz r6, VCPU_CR(r4) + mtcr r6 + mfspr r11, SPRN_SRR1 + LOAD_REG_IMMEDIATE(r0, UV_RETURN) + ld r7, VCPU_GPR(R7)(r4) + ld r6, VCPU_GPR(R6)(r4) + ld r4, VCPU_GPR(R4)(r4) + sc 2 /* * Enter the guest on a P9 or later system where we have exactly
@@ -3318,13 +3340,16 @@ END_MMU_FTR_SECTION_IFSET(MMU_FTR_TYPE_RADIX) * r0 is used as a scratch register */ kvmppc_msr_interrupt: + andis. r0, r11, MSR_S@h rldicl r0, r11, 64 - MSR_TS_S_LG, 62 - cmpwi r0, 2 /* Check if we are in transactional state.. */ + cmpwi cr1, r0, 2 /* Check if we are in transactional state.. */ ld r11, VCPU_INTR_MSR(r9) - bne 1f + bne cr1, 1f /* ... if transactional, change to suspended */ li r0, 1 1: rldimi r11, r0, MSR_TS_S_LG, 63 - MSR_TS_T_LG + beqlr + oris r11, r11, MSR_S@h /* preserve MSR_S bit setting */ blr /*
Signature · 1 line
2.20.1
Re: [PATCH v3 8/9] KVM: PPC: Ultravisor: Enter a secure guest
Quoted 18 lines · Claudio Carvalho
From: Sukadev Bhattiprolu <[email protected]>
To enter a secure guest, we have to go through the ultravisor, therefore we do a ucall when we are entering a secure guest.
This change is needed for any sort of entry to the secure guest from the hypervisor, whether it is a return from an hcall, a return from a hypervisor interrupt, or the first time that a secure guest vCPU is run.
If we are returning from an hcall, the results are already in the appropriate registers (R3:12), except for R6,7, which need to be restored before doing the ucall (
UV_RETURN).Have
fast_guest_returncheck the kvm_arch.secure_guest field so that a new CPU enters UV when started (in response to a RTAS start-cpu call).Thanks to input from Paul Mackerras, Ram Pai and Mike Anderson.
Quoted 8 lines
Signed-off-by Sukadev Bhattiprolu <[email protected]>[Pass SRR1 in r11 for
UV_RETURN, fixkvmppc_msr_interruptto preserve
theMSR_Sbit]Signed-off-by Paul Mackerras <[email protected]>[Fix UV_RETURN token number and arch.secure_guest check]Signed-off-by Ram Pai <[email protected]>[Update commit message and ret_to_ultra comment]Signed-off-by Claudio Carvalho <[email protected]>
Paul.
PATCH v3 9/9 KVM: PPC: Ultravisor: Check for MSR_S during hv_reset_msr
In reply to Claudio CarvalhoFrom: Michael Anderson <[email protected]>
- Check for MSR_S so that kvmppc_set_msr will include. Prior to this change return to guest would not have the S bit set.
- Patch based on comment from Paul Mackerras <[email protected]>
--- arch/powerpc/kvm/book3s_64_mmu_hv.c | 1 + 1 file changed, 1 insertion(+)
arch/powerpc/kvm/book3s_64_mmu_hv.c +1 -0
@@ -295,6 +295,7 @@ static void kvmppc_mmu_book3s_64_hv_reset_msr(struct kvm_vcpu *vcpu) msr |=MSR_TS_S ; else msr |= vcpu->arch.shregs.msr & MSR_TS_MASK; + msr |= vcpu->arch.shregs.msr &MSR_S ;kvmppc_set_msr (vcpu, msr); }
Signature · 1 line
2.20.1
Re: [PATCH v3 9/9] KVM: PPC: Ultravisor: Check for MSR_S during hv_reset_msr
Quoted 3 lines · Claudio Carvalho
From: Michael Anderson <[email protected]>
- Check for
MSR_Sso thatkvmppc_set_msrwill include. Prior to this
Will include what? "it" maybe?
Quoted 6 lines
change return to guest would not have the S bit set.
- Patch based on comment from Paul Mackerras <[email protected]>
Signed-off-by Michael Anderson <[email protected]>Signed-off-by Claudio Carvalho <[email protected]>
but you should reword the commit message fix that first sentence.
Paul.