KVM: s390: pv: implement lazy destroy for reboot
Subsystems (5)
Claudio Imbrenda
maintainerv12 cover KVM: s390: pv: implement lazy destroy for reboot
Previously, when a protected VM was rebooted or when it was shut down, its memory was made unprotected, and then the protected VM itself was destroyed. Looping over the whole address space can take some time, considering the overhead of the various Ultravisor Calls (UVCs). This means that a reboot or a shutdown would take a potentially long amount of time, depending on the amount of used memory.
This patchseries implements a deferred destroy mechanism for protected guests. When a protected guest is destroyed, its memory can be cleared in background, allowing the guest to restart or terminate significantly faster than before.
There are 2 possibilities when a protected VM is torn down:
* it still has an address space associated (reboot case)
* it does not have an address space anymore (shutdown case)
For the reboot case, two new commands are available for the KVM_S390_PV_COMMAND:
KVM_PV_ASYNC_CLEANUP_PREPARE: prepares the current protected VM for asynchronous teardown. The current VM will then continue immediately as non-protected. If a protected VM had already been set aside without starting the teardown process, this call will fail. In this case the userspace process should issue a normal KVM_PV_DISABLE
KVM_PV_ASYNC_CLEANUP_PERFORM: tears down the protected VM previously set aside for asychronous teardown. This PV command should ideally be issued by userspace from a separate thread. If a fatal signal is received (or the process terminates naturally), the command will terminate immediately without completing. The rest of the normal KVM teardown process will take care of properly cleaning up all leftovers.
The idea is that userspace should first issue the KVM_PV_ASYNC_CLEANUP_PREPARE command, and in case of success, create a new thread and issue KVM_PV_ASYNC_CLEANUP_PERFORM from there. This also allows for proper accounting of the CPU time needed for the asynchronous teardown.
This means that the same address space can have memory belonging to more than one protected guest, although only one will be running, the others will in fact not even have any CPUs.
The shutdown case should be dealt with in userspace (e.g. using clone(CLONE_VM)).
A module parameter is also provided to disable the new functionality, which is otherwise enabled by default. This should not be an issue since the new functionality is opt-in anyway. This is mainly thought to aid debugging.
v11->v12
* rebase
* fix and improve comments and documentation
* introduce the module parameter at the end of the series
* merge old patch 14 and 15
* minor cosmetic changes to improve readability
* renamed some functions and constants (including in uapi)
* use the lock instead of xchg
* kvm_s390_pv_deinit_cleanup_all will now cleanup all leftovers, while kvm_s390_pv_deinit will destroy the current protected VM without further action
* kvm_s390_pv_deinit_cleanup_all now called unconditionally when KVM is being torn down
* fix small memory leak if kvm_s390_pv_deinit_vm_fast fails
v10->v11
* rebase
* improve comments and patch descriptions
* rename s390_remove_old_asce to s390_unlist_old_asce
* rename DESTROY_LOOP_THRESHOLD to GATHER_GET_PAGES
* rename module parameter lazy_destroy to async_destroy
* move the WRITE_ONCE to be right after the UVC in patch 13
* improve handling leftover secure VMs in patch 14
* lock only when needed in patch 15, instead of always locking and then unlocking and locking again
* refactor should_export_before_import to make it more readable
v9->v10
* improved and expanded comments, fix typos
* add new patch: perform destroy configuration UVC before clearing memory for unconditional deinit_vm (instead of afterwards)
* explicitly initialize kvm->arch.pv.async_deinit in kvm_arch_init_vm
* do not try to call the destroy fast UVC in the MMU notifier if it is not available
v8->v9
* rebased
* added dependency on MMU_NOTIFIER for KVM in arch/s390/kvm/Kconfig
* add support for the Destroy Secure Configuration Fast UVC
* minor fixes
v7->v8
* switched patches 8 and 9
* improved comments, documentation and patch descriptions
* remove mm notifier when the struct kvm is torn down
* removed useless locks in the mm notifier
* use _ASCE_ORIGIN instead of PAGE_MASK for ASCEs
* cleanup of some compiler warnings
* remove some harmless but useless duplicate code
* the last parameter of __s390_uv_destroy_range is now bool
* rename the KVM capability to KVM_CAP_S390_PROTECTED_ASYNC_DISABLE
v6->v7
* moved INIT_LIST_HEAD inside spinlock in patch 1
* improved commit messages in patch 2
* added missing locks in patch 3
* added and expanded some comments in patch 11
* rebased
v5->v6
* completely reworked the series
* removed kernel thread for asynchronous teardown
* added new commands to KVM_S390_PV_COMMAND ioctl
v4->v5
* fixed and improved some patch descriptions
* added some comments to better explain what's going on
* use vma_lookup instead of find_vma
* rename is_protected to protected_count since now it's used as a counter
v3->v4
* added patch 2
* split patch 3
* removed the shutdown part -- will be a separate patchseries
* moved the patch introducing the module parameter
v2->v3
* added definitions for CC return codes for the UVC instruction
* improved make_secure_pte:
- renamed rc to cc
- added comments to explain why returning -EAGAIN is ok
* fixed kvm_s390_pv_replace_asce and kvm_s390_pv_remove_old_asce:
- renamed
- added locking
- moved to gmap.c
* do proper error management in do_secure_storage_access instead of trying again hoping to get a different exception
* fix outdated patch descriptions
v1->v2
* rebased on a more recent kernel
* improved/expanded some patch descriptions
* improves/expanded some comments
* added patch 1, which prevents stall notification when the system is under heavy load.
* rename some members of struct deferred_priv to improve readability
* avoid an use-after-free bug of the struct mm in case of shutdown
* add missing return when lazy destroy is disabled
* add support for OOM notifier
Claudio Imbrenda (18):
KVM: s390: pv: leak the topmost page table when destroy fails
KVM: s390: pv: handle secure storage violations for protected guests
KVM: s390: pv: handle secure storage exceptions for normal guests
KVM: s390: pv: refactor s390_reset_acc
KVM: s390: pv: usage counter instead of flag
KVM: s390: pv: add export before import
KVM: s390: pv: clear the state without memset
KVM: s390: pv: Add kvm_s390_cpus_from_pv to kvm-s390.h and add
documentation
KVM: s390: pv: add mmu_notifier
s390/mm: KVM: pv: when tearing down, try to destroy protected pages
KVM: s390: pv: refactoring of kvm_s390_pv_deinit_vm
KVM: s390: pv: destroy the configuration before its memory
KVM: s390: pv: asynchronous destroy for reboot
KVM: s390: pv: api documentation for asynchronous destroy
KVM: s390: pv: add KVM_CAP_S390_PROTECTED_ASYNC_DISABLE
KVM: s390: pv: avoid export before import if possible
KVM: s390: pv: support for Destroy fast UVC
KVM: s390: pv: module parameter to fence asynchronous destroy
Documentation/virt/kvm/api.rst | 31 ++- arch/s390/include/asm/gmap.h | 39 ++- arch/s390/include/asm/kvm_host.h | 4 + arch/s390/include/asm/mmu.h | 2 +- arch/s390/include/asm/mmu_context.h | 2 +- arch/s390/include/asm/pgtable.h | 21 +- arch/s390/include/asm/uv.h | 11 + arch/s390/kernel/uv.c | 85 ++++++ arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/kvm-s390.c | 101 +++++++- arch/s390/kvm/kvm-s390.h | 4 + arch/s390/kvm/pv.c | 384 +++++++++++++++++++++++++++- arch/s390/mm/fault.c | 23 +- arch/s390/mm/gmap.c | 177 +++++++++++-- include/uapi/linux/kvm.h | 3 + 15 files changed, 825 insertions(+), 63 deletions(-)
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 1/18 KVM: s390: pv: leak the topmost page table when destroy fails
Each secure guest must have a unique ASCE (address space control element); we must avoid that new guests use the same page for their ASCE, to avoid errors.
Since the ASCE mostly consists of the address of the topmost page table (plus some flags), we must not return that memory to the pool unless the ASCE is no longer in use.
Only a successful Destroy Secure Configuration UVC will make the ASCE reusable again.
If the Destroy Configuration UVC fails, the ASCE cannot be reused for a secure guest (either for the ASCE or for other memory areas). To avoid a collision, it must not be used again. This is a permanent error and the page becomes in practice unusable, so we set it aside and leak it. On failure we already leak other memory that belongs to the ultravisor (i.e. the variable and base storage for a guest) and not leaking the topmost page table was an oversight.
This error (and thus the leakage) should not happen unless the hardware is broken or KVM has some unknown serious bug.
--- arch/s390/include/asm/gmap.h | 2 + arch/s390/kvm/pv.c | 9 ++-- arch/s390/mm/gmap.c | 86 ++++++++++++++++++++++++++++++++++++ 3 files changed, 94 insertions(+), 3 deletions(-)
arch/s390/include/asm/gmap.h +2 -0
@@ -148,4 +148,6 @@ void gmap_sync_dirty_log_pmd(struct gmap *gmap, unsigned long dirty_bitmap[4], unsigned long gaddr, unsigned long vmaddr);int gmap_mark_unmergeable (void );void s390_reset_acc (struct mm_struct *mm); +void s390_unlist_old_asce (struct gmap *gmap); +int s390_replace_asce (struct gmap *gmap);#endif /* _ASM_S390_GMAP_H */
arch/s390/kvm/pv.c +6 -3
@@ -161,10 +161,13 @@ int kvm_s390_pv_deinit_vm(struct kvm *kvm, u16 *rc, u16 *rrc)atomic_set (&kvm->mm->context.is_protected,0 );KVM_UV_EVENT (kvm,3 ,"PROTVIRT DESTROY VM: rc %x rrc %x" , *rc, *rrc);WARN_ONCE (cc,"protvirt destroy vm failed rc %x rrc %x" , *rc, *rrc); -/* Inteded memory leak on "impossible" error */ -if (!cc) +/* Intended memory leak on "impossible" error */ +if (!cc) {kvm_s390_pv_dealloc_vm (kvm); -return cc ? -EIO :0 ; +return 0 ; + } +s390_replace_asce (kvm->arch.gmap); +return -EIO ; }int kvm_s390_pv_init_vm (struct kvm *kvm,u16 *rc,u16 *rrc)
arch/s390/mm/gmap.c +86 -0
89 lines not on this page · raw message
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 2/18 KVM: s390: pv: handle secure storage violations for protected guests
In reply to Claudio ImbrendaA secure storage violation is triggered when a protected guest tries to access secure memory that has been mapped erroneously, or that belongs to a different protected guest or to the ultravisor.
With upcoming patches, protected guests will be able to trigger secure storage violations in normal operation. This happens for example if a protected guest is rebooted with deferred destroy enabled and the new guest is also protected.
When the new protected guest touches pages that have not yet been destroyed, and thus are accounted to the previous protected guest, a secure storage violation is raised.
This patch adds handling of secure storage violations for protected guests.
This exception is handled by first trying to destroy the page, because it is expected to belong to a defunct protected guest where a destroy should be possible. Note that a secure page can only be destroyed if its protected VM does not have any CPUs, which only happens when the protected VM is being terminated. If that fails, a normal export of the page is attempted.
This means that pages that trigger the exception will be made non-secure (in one way or another) before attempting to use them again for a different secure guest.
--- arch/s390/include/asm/uv.h | 1 + arch/s390/kernel/uv.c | 55 ++++++++++++++++++++++++++++++++++++++ arch/s390/mm/fault.c | 10 +++++++ 3 files changed, 66 insertions(+)
arch/s390/include/asm/uv.h +1 -0
@@ -378,6 +378,7 @@ static inline int is_prot_virt_host(void) }int gmap_make_secure (struct gmap *gmap,unsigned long gaddr,void *uvcb); +int gmap_destroy_page (struct gmap *gmap,unsigned long gaddr);int uv_destroy_owned_page (unsigned long paddr);int uv_convert_from_secure (unsigned long paddr);int uv_convert_owned_from_secure (unsigned long paddr);
arch/s390/kernel/uv.c +55 -0
61 lines not on this page · raw message
arch/s390/mm/fault.c +10 -0
@@ -837,6 +837,16 @@ NOKPROBE_SYMBOL(do_non_secure_storage_access);void do_secure_storage_violation (struct pt_regs *regs) { +unsigned long gaddr = regs->int_parm_long & __FAIL_ADDR_MASK; +struct gmap *gmap = (struct gmap *)S390_lowcore.gmap; + +/* +* If the VM has been rebooted, its address space might still contain +* secure pages from the previous boot. +* Clear the page so it can be reused. +*/ +if (!gmap_destroy_page (gmap, gaddr)) +return ;/* * Either KVM messed up the secure guest mapping or the same * page is mapped into multiple secure guests.
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 3/18 KVM: s390: pv: handle secure storage exceptions for normal guests
In reply to Claudio ImbrendaWith upcoming patches, normal guests might touch secure pages.
This patch extends the existing exception handler to convert the pages to non secure also when the exception is triggered by a normal guest.
This can happen for example when a secure guest reboots; the first stage of a secure guest is non secure, and in general a secure guest can reboot into non-secure mode.
If the secure memory of the previous boot has not been cleared up completely yet (which will be allowed to happen in an upcoming patch), a non-secure guest might touch secure memory, which will need to be handled properly.
This means that gmap faults must be handled and not cause termination of the process. The handling is the same as userspace accesses, it's enough to translate the gmap address to a user address and then let the normal user fault code handle it.
--- arch/s390/mm/fault.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)
arch/s390/mm/fault.c +12 -1
@@ -754,6 +754,7 @@ void do_secure_storage_access(struct pt_regs *regs)struct vm_area_struct *vma;struct mm_struct *mm;struct page *page; +struct gmap *gmap;int rc;/*
@@ -783,6 +784,17 @@ void do_secure_storage_access(struct pt_regs *regs) }switch (get_fault_type (regs)) { +case GMAP_FAULT : + mm = current->mm; + gmap = (struct gmap *)S390_lowcore.gmap; +mmap_read_lock (mm); + addr =__gmap_translate (gmap, addr); +mmap_read_unlock (mm); +if (IS_ERR_VALUE (addr)) { +do_fault_error (regs,VM_ACCESS_FLAGS ,VM_FAULT_BADMAP ); +break ; + } + fallthrough;case USER_FAULT : mm = current->mm;mmap_read_lock (mm);
@@ -811,7 +823,6 @@ void do_secure_storage_access(struct pt_regs *regs)if (rc)BUG ();break ; -case GMAP_FAULT :default :do_fault_error (regs,VM_READ |VM_WRITE ,VM_FAULT_BADMAP );WARN_ON_ONCE (1 );
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 4/18 KVM: s390: pv: refactor s390_reset_acc
In reply to Claudio ImbrendaRefactor s390_reset_acc so that it can be reused in upcoming patches.
We don't want to hold all the locks used in a walk_page_range for too long, and the destroy page UVC does take some time to complete. Therefore we quickly gather the pages to destroy, and then destroy them without holding all the locks.
The new refactored function optionally allows to return early without completing if a fatal signal is pending (and return and appropriate error code). Two wrappers are provided to call the new function.
--- arch/s390/include/asm/gmap.h | 37 +++++++++++++- arch/s390/kvm/pv.c | 12 ++++- arch/s390/mm/gmap.c | 99 +++++++++++++++++++++++++----------- 3 files changed, 116 insertions(+), 32 deletions(-)
arch/s390/include/asm/gmap.h +36 -1
@@ -147,7 +147,42 @@ int gmap_mprotect_notify(struct gmap *, unsigned long start,void gmap_sync_dirty_log_pmd (struct gmap *gmap,unsigned long dirty_bitmap[4 ],unsigned long gaddr,unsigned long vmaddr);int gmap_mark_unmergeable (void ); -void s390_reset_acc (struct mm_struct *mm);void s390_unlist_old_asce (struct gmap *gmap);int s390_replace_asce (struct gmap *gmap); +void s390_uv_destroy_pfns (unsigned long count,unsigned long *pfns); +int __s390_uv_destroy_range (struct mm_struct *mm,unsigned long start, +unsigned long end,bool interruptible); + +/** +* s390_uv_destroy_range - Destroy a range of pages in the given mm. +* @mm: the mm on which to operate on +* @start: the start of the range +* @end: the end of the range +* +* This function will call cond_sched, so it should not generate stalls, but +* it will otherwise only return when it completed. +*/ +static inline void s390_uv_destroy_range (struct mm_struct *mm,unsigned long start, +unsigned long end) +{ + (void )__s390_uv_destroy_range (mm, start, end, false); +} + +/** +* s390_uv_destroy_range_interruptible - Destroy a range of pages in the +* given mm, but stop when a fatal signal is received. +* @mm: the mm on which to operate on +* @start: the start of the range +* @end: the end of the range +* +* This function will call cond_sched, so it should not generate stalls. If +* a fatal signal is received, it will return with -EINTR immediately, +* without finishing destroying the whole range. Upon successful +* completion, 0 is returned. +*/ +static inline int s390_uv_destroy_range_interruptible (struct mm_struct *mm,unsigned long start, +unsigned long end) +{ +return __s390_uv_destroy_range (mm, start, end, true); +}#endif /* _ASM_S390_GMAP_H */
arch/s390/kvm/pv.c +10 -2
@@ -12,6 +12,8 @@#include <asm/gmap.h> #include <asm/uv.h> #include <asm/mman.h> +#include <linux/pagewalk.h> +#include <linux/sched/mm.h> #include "kvm-s390.h" int kvm_s390_pv_destroy_cpu (struct kvm_vcpu *vcpu,u16 *rc,u16 *rrc)
@@ -152,8 +154,14 @@ int kvm_s390_pv_deinit_vm(struct kvm *kvm, u16 *rc, u16 *rrc) {int cc; -/* make all pages accessible before destroying the guest */ -s390_reset_acc (kvm->mm); +/* +* if the mm still has a mapping, make all its pages accessible +* before destroying the guest +*/ +if (mmget_not_zero (kvm->mm)) { +s390_uv_destroy_range (kvm->mm,0 ,TASK_SIZE ); +mmput (kvm->mm); + } cc =uv_cmd_nodata (kvm_s390_pv_get_handle (kvm),UVC_CMD_DESTROY_SEC_CONF , rc, rrc);
arch/s390/mm/gmap.c +70 -29
114 lines not on this page · raw message
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 5/18 KVM: s390: pv: usage counter instead of flag
In reply to Claudio ImbrendaUse the new protected_count field as a counter instead of the old is_protected flag. This will be used in upcoming patches.
Increment the counter when a secure configuration is created, and decrement it when it is destroyed. Previously the flag was set when the set secure parameters UVC was performed.
--- arch/s390/include/asm/mmu.h | 2 +- arch/s390/include/asm/mmu_context.h | 2 +- arch/s390/include/asm/pgtable.h | 2 +- arch/s390/kvm/pv.c | 12 +++++++----- 4 files changed, 10 insertions(+), 8 deletions(-)
arch/s390/include/asm/mmu.h +1 -1
@@ -18,7 +18,7 @@ typedef struct {unsigned long asce_limit;unsigned long vdso_base;/* The mmu context belongs to a secure guest. */ -atomic_t is_protected; +atomic_t protected_count;/* * The following bitfields need a down_write on the mm * semaphore when they are written to. As they are only
arch/s390/include/asm/mmu_context.h +1 -1
@@ -26,7 +26,7 @@ static inline int init_new_context(struct task_struct *tsk,INIT_LIST_HEAD (&mm->context.gmap_list);cpumask_clear (&mm->context.cpu_attach_mask);atomic_set (&mm->context.flush_count,0 ); -atomic_set (&mm->context.is_protected,0 ); +atomic_set (&mm->context.protected_count,0 ); mm->context.gmap_asce =0 ; mm->context.flush_mm =0 ;#ifdef CONFIG_PGSTE
arch/s390/include/asm/pgtable.h +1 -1
@@ -525,7 +525,7 @@ static inline int mm_has_pgste(struct mm_struct *mm)static inline int mm_is_protected (struct mm_struct *mm) {#ifdef CONFIG_PGSTE -if (unlikely (atomic_read (&mm->context.is_protected))) +if (unlikely (atomic_read (&mm->context.protected_count)))return 1 ;#endif return 0 ;
arch/s390/kvm/pv.c +7 -5
@@ -166,7 +166,8 @@ int kvm_s390_pv_deinit_vm(struct kvm *kvm, u16 *rc, u16 *rrc) cc =uv_cmd_nodata (kvm_s390_pv_get_handle (kvm),UVC_CMD_DESTROY_SEC_CONF , rc, rrc);WRITE_ONCE (kvm->arch.gmap->guest_handle,0 ); -atomic_set (&kvm->mm->context.is_protected,0 ); +if (!cc) +atomic_dec (&kvm->mm->context.protected_count);KVM_UV_EVENT (kvm,3 ,"PROTVIRT DESTROY VM: rc %x rrc %x" , *rc, *rrc);WARN_ONCE (cc,"protvirt destroy vm failed rc %x rrc %x" , *rc, *rrc);/* Intended memory leak on "impossible" error */
@@ -208,11 +209,14 @@ int kvm_s390_pv_init_vm(struct kvm *kvm, u16 *rc, u16 *rrc)/* Outputs */ kvm->arch.pv.handle = uvcb.guest_handle; +atomic_inc (&kvm->mm->context.protected_count);if (cc) { -if (uvcb.header.rc &UVC_RC_NEED_DESTROY ) +if (uvcb.header.rc &UVC_RC_NEED_DESTROY ) {kvm_s390_pv_deinit_vm (kvm, &dummy, &dummy); -else + }else { +atomic_dec (&kvm->mm->context.protected_count);kvm_s390_pv_dealloc_vm (kvm); + }return -EIO ; } kvm->arch.gmap->guest_handle = uvcb.guest_handle;
@@ -235,8 +239,6 @@ int kvm_s390_pv_set_sec_parms(struct kvm *kvm, void *hdr, u64 length, u16 *rc, *rrc = uvcb.header.rrc;KVM_UV_EVENT (kvm,3 ,"PROTVIRT VM SET PARMS: rc %x rrc %x" , *rc, *rrc); -if (!cc) -atomic_set (&kvm->mm->context.is_protected,1 );return cc ? -EINVAL :0 ; }
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 6/18 KVM: s390: pv: add export before import
In reply to Claudio ImbrendaDue to upcoming changes, it will be possible to temporarily have multiple protected VMs in the same address space, although only one will be actually active.
In that scenario, it is necessary to perform an export of every page that is to be imported, since the hardware does not allow a page belonging to a protected guest to be imported into a different protected guest.
This also applies to pages that are shared, and thus accessible by the host.
--- arch/s390/kernel/uv.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)
arch/s390/kernel/uv.c +28 -0
@@ -234,6 +234,32 @@ static int make_secure_pte(pte_t *ptep, unsigned long addr,return uvcb->rc ==0x10a ? -ENXIO : -EINVAL ; } +/** +* should_export_before_import - Determine whether an export is needed +* before an import-like operation +* @uvcb: the Ultravisor control block of the UVC to be performed +* @mm: the mm of the process +* +* Returns whether an export is needed before every import-like operation. +* This is needed for shared pages, which don't trigger a secure storage +* exception when accessed from a different guest. +* +* Although considered as one, the Unpin Page UVC is not an actual import, +* so it is not affected. +* +* No export is needed also when there is only one protected VM, because the +* page cannot belong to the wrong VM in that case (there is no "other VM" +* it can belong to). +* +* Return: true if an export is needed before every import, otherwise false. +*/ +static bool should_export_before_import (struct uv_cb_header *uvcb,struct mm_struct *mm) +{ +if (uvcb->cmd ==UVC_CMD_UNPIN_PAGE_SHARED ) +return false; +return atomic_read (&mm->context.protected_count) >1 ; +} +/* * Requests the Ultravisor to make a page accessible to a guest. * If it's brought in the first time, it will be cleared. If
@@ -277,6 +303,8 @@ int gmap_make_secure(struct gmap *gmap, unsigned long gaddr, void *uvcb)lock_page (page); ptep =get_locked_pte (gmap->mm, uaddr, &ptelock); +if (should_export_before_import (uvcb, gmap->mm)) +uv_convert_from_secure (page_to_phys (page)); rc =make_secure_pte (ptep, uaddr, page, uvcb);pte_unmap_unlock (ptep, ptelock);unlock_page (page);
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 7/18 KVM: s390: pv: clear the state without memset
In reply to Claudio ImbrendaDo not use memset to clean the whole struct kvm_s390_pv; instead, explicitly clear the fields that need to be cleared.
Upcoming patches will introduce new fields in the struct kvm_s390_pv that will not need to be cleared.
--- arch/s390/kvm/pv.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
arch/s390/kvm/pv.c +9 -1
@@ -16,6 +16,14 @@#include <linux/sched/mm.h> #include "kvm-s390.h" +static void kvm_s390_clear_pv_state (struct kvm *kvm) +{ + kvm->arch.pv.handle =0 ; + kvm->arch.pv.guest_len =0 ; + kvm->arch.pv.stor_base =0 ; + kvm->arch.pv.stor_var =NULL ; +} +int kvm_s390_pv_destroy_cpu (struct kvm_vcpu *vcpu,u16 *rc,u16 *rrc) {int cc;
@@ -110,7 +118,7 @@ static void kvm_s390_pv_dealloc_vm(struct kvm *kvm)vfree (kvm->arch.pv.stor_var);free_pages (kvm->arch.pv.stor_base,get_order (uv_info.guest_base_stor_len)); -memset (&kvm->arch.pv,0 ,sizeof (kvm->arch.pv)); +kvm_s390_clear_pv_state (kvm); }static int kvm_s390_pv_alloc_vm (struct kvm *kvm)
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 8/18 KVM: s390: pv: Add kvm_s390_cpus_from_pv to kvm-s390.h and add documentation
In reply to Claudio ImbrendaFuture changes make it necessary to call this function from pv.c.
While we are at it, let's properly document kvm_s390_cpus_from_pv() and kvm_s390_cpus_to_pv().
--- arch/s390/kvm/kvm-s390.c | 36 ++++++++++++++++++++++++++++++------ arch/s390/kvm/kvm-s390.h | 1 + 2 files changed, 31 insertions(+), 6 deletions(-)
arch/s390/kvm/kvm-s390.c +30 -6
@@ -2166,12 +2166,25 @@ static int kvm_s390_set_cmma_bits(struct kvm *kvm,return r; } -static int kvm_s390_cpus_from_pv (struct kvm *kvm,u16 *rcp,u16 *rrcp) +/** +* kvm_s390_cpus_from_pv - Convert all protected vCPUs in a protected VM to +* non protected. +* @kvm: the VM whose protected vCPUs are to be converted +* @rc: return value for the RC field of the UVC (in case of error) +* @rrc: return value for the RRC field of the UVC (in case of error) +* +* Does not stop in case of error, tries to convert as many +* CPUs as possible. In case of error, the RC and RRC of the last error are +* returned. +* +* Return: 0 in case of success, otherwise -EIO +*/ +int kvm_s390_cpus_from_pv (struct kvm *kvm,u16 *rc,u16 *rrc) {struct kvm_vcpu *vcpu; -u16 rc, rrc; -int ret =0 ;unsigned long i; +u16 _rc, _rrc; +int ret =0 ;/* * We ignore failures and try to destroy as many CPUs as possible.
@@ -2183,9 +2196,9 @@ static int kvm_s390_cpus_from_pv(struct kvm *kvm, u16 *rcp, u16 *rrcp)*/ kvm_for_each_vcpu (i, vcpu, kvm) {mutex_lock (&vcpu->mutex); -if (kvm_s390_pv_destroy_cpu (vcpu, &rc, &rrc) && !ret) { - *rcp = rc; - *rrcp = rrc; +if (kvm_s390_pv_destroy_cpu (vcpu, &_rc, &_rrc) && !ret) { + *rc = _rc; + *rrc = _rrc; ret = -EIO ; }mutex_unlock (&vcpu->mutex);
@@ -2196,6 +2209,17 @@ static int kvm_s390_cpus_from_pv(struct kvm *kvm, u16 *rcp, u16 *rrcp)return ret; } +/** +* kvm_s390_cpus_to_pv - Convert all non-protected vCPUs in a protected VM +* to protected. +* @kvm: the VM whose protected vCPUs are to be converted +* @rc: return value for the RC field of the UVC (in case of error) +* @rrc: return value for the RRC field of the UVC (in case of error) +* +* Tries to undo the conversion in case of error. +* +* Return: 0 in case of success, otherwise -EIO +*/ static int kvm_s390_cpus_to_pv (struct kvm *kvm,u16 *rc,u16 *rrc) {unsigned long i;
arch/s390/kvm/kvm-s390.h +1 -0
@@ -374,6 +374,7 @@ int kvm_s390_vcpu_setup_cmma(struct kvm_vcpu *vcpu);void kvm_s390_vcpu_unsetup_cmma (struct kvm_vcpu *vcpu);void kvm_s390_set_cpu_timer (struct kvm_vcpu *vcpu,__u64 cputm);__u64 kvm_s390_get_cpu_timer (struct kvm_vcpu *vcpu); +int kvm_s390_cpus_from_pv (struct kvm *kvm,u16 *rc,u16 *rrc);/* implemented in diag.c */ int kvm_s390_handle_diag (struct kvm_vcpu *vcpu);
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 9/18 KVM: s390: pv: add mmu_notifier
In reply to Claudio ImbrendaAdd an mmu_notifier for protected VMs. The callback function is triggered when the mm is torn down, and will attempt to convert all protected vCPUs to non-protected. This allows the mm teardown to use the destroy page UVC instead of export.
Also make KVM select CONFIG_MMU_NOTIFIER, needed to use mmu_notifiers.
--- arch/s390/include/asm/kvm_host.h | 2 ++ arch/s390/kvm/Kconfig | 1 + arch/s390/kvm/kvm-s390.c | 10 ++++++++++ arch/s390/kvm/pv.c | 26 ++++++++++++++++++++++++++ 4 files changed, 39 insertions(+)
arch/s390/include/asm/kvm_host.h +2 -0
@@ -19,6 +19,7 @@#include <linux/kvm.h> #include <linux/seqlock.h> #include <linux/module.h> +#include <linux/mmu_notifier.h> #include <asm/debug.h> #include <asm/cpu.h> #include <asm/fpu/api.h>
@@ -923,6 +924,7 @@ struct kvm_s390_pv {u64 guest_len;unsigned long stor_base;void *stor_var; +struct mmu_notifier mmu_notifier; };struct kvm_arch {
arch/s390/kvm/Kconfig +1 -0
@@ -34,6 +34,7 @@ config KVM select SRCU select KVM_VFIO select INTERVAL_TREE + select MMU_NOTIFIER help Support hosting paravirtualized guest machines using the SIE virtualization capability on the mainframe. This should work
arch/s390/kvm/kvm-s390.c +10 -0
@@ -31,6 +31,7 @@#include <linux/sched/signal.h> #include <linux/string.h> #include <linux/pgtable.h> +#include <linux/mmu_notifier.h> #include <asm/asm-offsets.h> #include <asm/lowcore.h>
@@ -2928,6 +2929,15 @@ void kvm_arch_destroy_vm(struct kvm *kvm)*/ if (kvm_s390_pv_get_handle (kvm))kvm_s390_pv_deinit_vm (kvm, &rc, &rrc); +/* +* Remove the mmu notifier only when the whole KVM VM is torn down, +* and only if one was registered to begin with. If the VM is +* currently not protected, but has been previously been protected, +* then it's possible that the notifier is still registered. +*/ +if (kvm->arch.pv.mmu_notifier.ops) +mmu_notifier_unregister (&kvm->arch.pv.mmu_notifier, kvm->mm); +debug_unregister (kvm->arch.dbf);free_page ((unsigned long )kvm->arch.sie_page2);if (!kvm_is_ucontrol (kvm))
arch/s390/kvm/pv.c +26 -0
@@ -14,6 +14,7 @@#include <asm/mman.h> #include <linux/pagewalk.h> #include <linux/sched/mm.h> +#include <linux/mmu_notifier.h> #include "kvm-s390.h" static void kvm_s390_clear_pv_state (struct kvm *kvm)
@@ -187,6 +188,26 @@ int kvm_s390_pv_deinit_vm(struct kvm *kvm, u16 *rc, u16 *rrc)return -EIO ; } +static void kvm_s390_pv_mmu_notifier_release (struct mmu_notifier *subscription, +struct mm_struct *mm) +{ + struct kvm *kvm = container_of(subscription, struct kvm, arch.pv.mmu_notifier); +u16 dummy; + +/* +* No locking is needed since this is the last thread of the last user of this +* struct mm. +* When the struct kvm gets deinitialized, this notifier is also +* unregistered. This means that if this notifier runs, then the +* struct kvm is still valid. +*/ +kvm_s390_cpus_from_pv (kvm, &dummy, &dummy); +} + +static const struct mmu_notifier_ops kvm_s390_pv_mmu_notifier_ops = { + .release = kvm_s390_pv_mmu_notifier_release, +}; +int kvm_s390_pv_init_vm (struct kvm *kvm,u16 *rc,u16 *rrc) {struct uv_cb_cgc uvcb = {
@@ -228,6 +249,11 @@ int kvm_s390_pv_init_vm(struct kvm *kvm, u16 *rc, u16 *rrc)return -EIO ; } kvm->arch.gmap->guest_handle = uvcb.guest_handle; +/* Add the notifier only once. No races because we hold kvm->lock */ +if (kvm->arch.pv.mmu_notifier.ops != &kvm_s390_pv_mmu_notifier_ops) { + kvm->arch.pv.mmu_notifier.ops = &kvm_s390_pv_mmu_notifier_ops; +mmu_notifier_register (&kvm->arch.pv.mmu_notifier, kvm->mm); + }return 0 ; }
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 10/18 s390/mm: KVM: pv: when tearing down, try to destroy protected pages
In reply to Claudio ImbrendaWhen ptep_get_and_clear_full is called for a mm teardown, we will now attempt to destroy the secure pages. This will be faster than export.
In case it was not a teardown, or if for some reason the destroy page UVC failed, we try with an export page, like before.
--- arch/s390/include/asm/pgtable.h | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-)
arch/s390/include/asm/pgtable.h +16 -3
@@ -1182,9 +1182,22 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm, }else { res =ptep_xchg_lazy (mm, addr, ptep,__pte (_PAGE_INVALID)); } -/* At this point the reference through the mapping is still present */ -if (mm_is_protected (mm) &&pte_present (res)) -uv_convert_owned_from_secure (pte_val (res) &PAGE_MASK ); +/* Nothing to do */ +if (!mm_is_protected (mm) || !pte_present (res)) +return res; +/* +* At this point the reference through the mapping is still present. +* The notifier should have destroyed all protected vCPUs at this +* point, so the destroy should be successful. +*/ +if (full && !uv_destroy_owned_page (pte_val (res) &PAGE_MASK )) +return res; +/* +* If something went wrong and the page could not be destroyed, or +* if this is not a mm teardown, the slower export is used as +* fallback instead. +*/ +uv_convert_owned_from_secure (pte_val (res) &PAGE_MASK );return res; }
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 11/18 KVM: s390: pv: refactoring of kvm_s390_pv_deinit_vm
In reply to Claudio ImbrendaRefactor kvm_s390_pv_deinit_vm to improve readability and simplify the improvements that are coming in subsequent patches.
No functional change intended.
[note: this can potentially be squashed into the next patch, I factored it out to simplify the review process]
--- arch/s390/kvm/pv.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-)
arch/s390/kvm/pv.c +8 -8
@@ -175,17 +175,17 @@ int kvm_s390_pv_deinit_vm(struct kvm *kvm, u16 *rc, u16 *rrc) cc =uv_cmd_nodata (kvm_s390_pv_get_handle (kvm),UVC_CMD_DESTROY_SEC_CONF , rc, rrc);WRITE_ONCE (kvm->arch.gmap->guest_handle,0 ); -if (!cc) -atomic_dec (&kvm->mm->context.protected_count); -KVM_UV_EVENT (kvm,3 ,"PROTVIRT DESTROY VM: rc %x rrc %x" , *rc, *rrc); -WARN_ONCE (cc,"protvirt destroy vm failed rc %x rrc %x" , *rc, *rrc); -/* Intended memory leak on "impossible" error */ if (!cc) { +atomic_dec (&kvm->mm->context.protected_count);kvm_s390_pv_dealloc_vm (kvm); -return 0 ; + }else { +/* Intended memory leak on "impossible" error */ +s390_replace_asce (kvm->arch.gmap); } -s390_replace_asce (kvm->arch.gmap); -return -EIO ; +KVM_UV_EVENT (kvm,3 ,"PROTVIRT DESTROY VM: rc %x rrc %x" , *rc, *rrc); +WARN_ONCE (cc,"protvirt destroy vm failed rc %x rrc %x" , *rc, *rrc); + +return cc ? -EIO :0 ; } static void kvm_s390_pv_mmu_notifier_release(struct mmu_notifier *subscription,
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 12/18 KVM: s390: pv: destroy the configuration before its memory
In reply to Claudio ImbrendaMove the Destroy Secure Configuration UVC before the loop to destroy the memory. If the protected VM has memory, it will be cleaned up and made accessible by the Destroy Secure Configuration UVC. The struct page for the relevant pages will still have the protected bit set, so the loop is still needed to clean that up.
Switching the order of those two operations does not change the outcome, but it is significantly faster.
--- arch/s390/kvm/pv.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
arch/s390/kvm/pv.c +3 -3
@@ -163,6 +163,9 @@ int kvm_s390_pv_deinit_vm(struct kvm *kvm, u16 *rc, u16 *rrc) {int cc; + cc =uv_cmd_nodata (kvm_s390_pv_get_handle (kvm), +UVC_CMD_DESTROY_SEC_CONF , rc, rrc); +WRITE_ONCE (kvm->arch.gmap->guest_handle,0 );/* * if the mm still has a mapping, make all its pages accessible * before destroying the guest
@@ -172,9 +175,6 @@ int kvm_s390_pv_deinit_vm(struct kvm *kvm, u16 *rc, u16 *rrc)mmput (kvm->mm); } - cc =uv_cmd_nodata (kvm_s390_pv_get_handle (kvm), -UVC_CMD_DESTROY_SEC_CONF , rc, rrc); -WRITE_ONCE (kvm->arch.gmap->guest_handle,0 );if (!cc) {atomic_dec (&kvm->mm->context.protected_count);kvm_s390_pv_dealloc_vm (kvm);
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 13/18 KVM: s390: pv: asynchronous destroy for reboot
In reply to Claudio ImbrendaUntil now, destroying a protected guest was an entirely synchronous operation that could potentially take a very long time, depending on the size of the guest, due to the time needed to clean up the address space from protected pages.
This patch implements an asynchronous destroy mechanism, that allows a protected guest to reboot significantly faster than previously.
This is achieved by clearing the pages of the old guest in background. In case of reboot, the new guest will be able to run in the same address space almost immediately.
The old protected guest is then only destroyed when all of its memory has been destroyed or otherwise made non protected.
Two new PV commands are added for the KVM_S390_PV_COMMAND ioctl:
KVM_PV_ASYNC_CLEANUP_PREPARE: prepares the current protected VM for asynchronous teardown. The current VM will then continue immediately as non-protected. If a protected VM had already been set aside without starting the teardown process, this call will fail. There can be at most one prepared VM at any time.
KVM_PV_ASYNC_CLEANUP_PERFORM: tears down the protected VM previously set aside for asynchronous teardown. This PV command should ideally be issued by userspace from a separate thread. If a fatal signal is received (or the process terminates naturally), the command will terminate immediately without completing. The rest of the normal KVM teardown process will take care of properly cleaning up all remaining protected VMs.
--- arch/s390/include/asm/kvm_host.h | 2 + arch/s390/kvm/kvm-s390.c | 46 ++++- arch/s390/kvm/kvm-s390.h | 3 + arch/s390/kvm/pv.c | 290 +++++++++++++++++++++++++++++-- include/uapi/linux/kvm.h | 2 + 5 files changed, 325 insertions(+), 18 deletions(-)
arch/s390/include/asm/kvm_host.h +2 -0
@@ -924,6 +924,8 @@ struct kvm_s390_pv {u64 guest_len;unsigned long stor_base;void *stor_var; +void *set_aside; +struct list_head need_cleanup;struct mmu_notifier mmu_notifier; };
arch/s390/kvm/kvm-s390.c +38 -8
98 lines not on this page · raw message
arch/s390/kvm/kvm-s390.h +3 -0
@@ -243,6 +243,9 @@ static inline u32 kvm_s390_get_gisa_desc(struct kvm *kvm)/* implemented in pv.c */ int kvm_s390_pv_destroy_cpu (struct kvm_vcpu *vcpu,u16 *rc,u16 *rrc);int kvm_s390_pv_create_cpu (struct kvm_vcpu *vcpu,u16 *rc,u16 *rrc); +int kvm_s390_pv_set_aside (struct kvm *kvm,u16 *rc,u16 *rrc); +int kvm_s390_pv_deinit_aside_vm (struct kvm *kvm,u16 *rc,u16 *rrc); +int kvm_s390_pv_deinit_cleanup_all (struct kvm *kvm,u16 *rc,u16 *rrc);int kvm_s390_pv_deinit_vm (struct kvm *kvm,u16 *rc,u16 *rrc);int kvm_s390_pv_init_vm (struct kvm *kvm,u16 *rc,u16 *rrc); int kvm_s390_pv_set_sec_parms(struct kvm *kvm, void *hdr, u64 length, u16 *rc,
arch/s390/kvm/pv.c +280 -10
315 lines not on this page · raw message
include/uapi/linux/kvm.h +2 -0
@@ -1668,6 +1668,8 @@ enum pv_cmd_id {KVM_PV_VERIFY ,KVM_PV_PREP_RESET ,KVM_PV_UNSHARE_ALL , +KVM_PV_ASYNC_CLEANUP_PREPARE , + KVM_PV_ASYNC_CLEANUP_PERFORM, }; struct kvm_pv_cmd {
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 14/18 KVM: s390: pv: api documentation for asynchronous destroy
In reply to Claudio ImbrendaAdd documentation for the new commands added to the KVM_S390_PV_COMMAND ioctl.
--- Documentation/virt/kvm/api.rst | 31 ++++++++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-)
Documentation/virt/kvm/api.rst +28 -3
@@ -5143,11 +5143,13 @@ KVM_PV_ENABLE ===== ============================= KVM_PV_DISABLE - Deregister the VM from the Ultravisor and reclaim the memory that had been donated to the Ultravisor, making it usable by the kernel - again. All registered VCPUs are converted back to non-protected - ones. + again. All registered VCPUs are converted back to non-protected + ones. If a previous VM had been set aside for asynchonous teardown + with KVM_PV_ASYNC_CLEANUP_PREPARE and not actually torn down with + KVM_PV_ASYNC_CLEANUP_PERFORM, it will be torn down in this call + together with the current VM. KVM_PV_VM_SET_SEC_PARMS Pass the image header from VM memory to the Ultravisor in
@@ -5160,6 +5162,29 @@ KVM_PV_VM_VERIFY Verify the integrity of the unpacked image. Only if this succeeds, KVM is allowed to start protected VCPUs. +KVM_PV_ASYNC_CLEANUP_PREPARE + Prepare the current protected VM for asynchronous teardown. Most + resources used by the current protected VM will be set aside for a + subsequent asynchronous teardown. The current protected VM will then + resume execution immediately as non-protected. There can be at most + one protected VM set aside at any time. If a protected VM had + already been set aside without starting the asynchronous teardown + process, this call will fail. In that case, the userspace process + should issue a normal KVM_PV_DISABLE. The resources set aside with + this call will need to be cleaned up with a subsequent call to + KVM_PV_ASYNC_CLEANUP_PERFORM or KVM_PV_DISABLE, otherwise they will + be cleaned up when KVM terminates. + +KVM_PV_ASYNC_CLEANUP_PERFORM + Tear down the protected VM previously set aside with + KVM_PV_ASYNC_CLEANUP_PREPARE. The resources that had been set aside + will be freed during the execution of this command. This PV command + should ideally be issued by userspace from a separate thread. If a + fatal signal is received (or the process terminates naturally), the + command will terminate immediately without completing, and the normal + KVM shutdown procedure will take care of cleaning up all remaining + protected VMs. + 4.126 KVM_X86_SET_MSR_FILTER ----------------------------
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 15/18 KVM: s390: pv: add KVM_CAP_S390_PROTECTED_ASYNC_DISABLE
In reply to Claudio ImbrendaAdd KVM_CAP_S390_PROTECTED_ASYNC_DISABLE to signal that the KVM_PV_ASYNC_DISABLE and KVM_PV_ASYNC_DISABLE_PREPARE commands for the KVM_S390_PV_COMMAND ioctl are available.
--- arch/s390/kvm/kvm-s390.c | 3 +++ include/uapi/linux/kvm.h | 1 + 2 files changed, 4 insertions(+)
arch/s390/kvm/kvm-s390.c +3 -0
@@ -606,6 +606,9 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext)case KVM_CAP_S390_BPB : r =test_facility (82 );break ; +case KVM_CAP_S390_PROTECTED_ASYNC_DISABLE : + r = async_destroy &&is_prot_virt_host (); +break ;case KVM_CAP_S390_PROTECTED : r =is_prot_virt_host ();break ;
include/uapi/linux/kvm.h +1 -0
@@ -1157,6 +1157,7 @@ struct kvm_ppc_resize_hpt {#define KVM_CAP_VM_TSC_CONTROL 214#define KVM_CAP_SYSTEM_EVENT_DATA 215#define KVM_CAP_ARM_SYSTEM_SUSPEND 216 +#define KVM_CAP_S390_PROTECTED_ASYNC_DISABLE 225#ifdef KVM_CAP_IRQ_ROUTING
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 16/18 KVM: s390: pv: avoid export before import if possible
In reply to Claudio ImbrendaIf the appropriate UV feature bit is set, there is no need to perform an export before import.
--- arch/s390/kernel/uv.c | 2 ++ 1 file changed, 2 insertions(+)
arch/s390/kernel/uv.c +2 -0
@@ -255,6 +255,8 @@ static int make_secure_pte(pte_t *ptep, unsigned long addr,*/ static bool should_export_before_import (struct uv_cb_header *uvcb,struct mm_struct *mm) { +if (test_bit_inv (BIT_UV_FEAT_MISC , &uv_info.uv_feature_indications)) +return false;if (uvcb->cmd ==UVC_CMD_UNPIN_PAGE_SHARED )return false;return atomic_read (&mm->context.protected_count) >1 ;
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 17/18 KVM: s390: pv: support for Destroy fast UVC
In reply to Claudio ImbrendaAdd support for the Destroy Secure Configuration Fast Ultravisor call, and take advantage of it for asynchronous destroy.
When supported, the protected guest is destroyed immediately using the new UVC, leaving only the memory to be cleaned up asynchronously.
--- arch/s390/include/asm/uv.h | 10 +++++++ arch/s390/kvm/pv.c | 57 ++++++++++++++++++++++++++++++++------ 2 files changed, 59 insertions(+), 8 deletions(-)
arch/s390/include/asm/uv.h +10 -0
@@ -34,6 +34,7 @@#define UVC_CMD_INIT_UV 0x000f#define UVC_CMD_CREATE_SEC_CONF 0x0100#define UVC_CMD_DESTROY_SEC_CONF 0x0101 +#define UVC_CMD_DESTROY_SEC_CONF_FAST 0x0102#define UVC_CMD_CREATE_SEC_CPU 0x0120#define UVC_CMD_DESTROY_SEC_CPU 0x0121#define UVC_CMD_CONV_TO_SEC_STOR 0x0200
@@ -77,6 +78,7 @@ enum uv_cmds_inst {BIT_UVC_CMD_UNSHARE_ALL =20 ,BIT_UVC_CMD_PIN_PAGE_SHARED =21 ,BIT_UVC_CMD_UNPIN_PAGE_SHARED =22 , +BIT_UVC_CMD_DESTROY_SEC_CONF_FAST =23 , BIT_UVC_CMD_RETR_ATTEST = 28, };
@@ -213,6 +215,14 @@ struct uv_cb_nodata {u64 reserved20[4 ]; } __packed__aligned(8) ; +/* Destroy Configuration Fast */ +struct uv_cb_destroy_fast { +struct uv_cb_header header; +u64 reserved08[2 ]; +u64 handle; +u64 reserved20[5 ]; +} __packed__aligned(8) ; +/* Set Shared Access */ struct uv_cb_share {struct uv_cb_header header;
arch/s390/kvm/pv.c +49 -8
100 lines not on this page · raw message
Signature · 1 line
2.36.1
Claudio Imbrenda
maintainerPATCH v12 18/18 KVM: s390: pv: module parameter to fence asynchronous destroy
In reply to Claudio ImbrendaAdd the module parameter "async_destroy", to allow the asynchronous destroy mechanism to be switched off. This might be useful for debugging purposes.
The parameter is enabled by default since the feature is opt-in anyway.
--- arch/s390/kvm/kvm-s390.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
arch/s390/kvm/kvm-s390.c +7 -1
@@ -207,7 +207,13 @@ unsigned int diag9c_forwarding_hz;module_param (diag9c_forwarding_hz, uint,0644 );MODULE_PARM_DESC (diag9c_forwarding_hz,"Maximum diag9c forwarding per second, 0 to turn off" ); -static int async_destroy; +/* +* allow asynchronous deinit for protected guests; enable by default since +* the feature is opt-in anyway +*/ +static int async_destroy =1 ; +module_param (async_destroy, int,0444 ); +MODULE_PARM_DESC (async_destroy,"Asynchronous destroy for protected guests" );/* * For now we handle at most 16 double words as this is what the s390 base
Signature · 1 line
2.36.1
Janosch Frank
maintainerQuoted 7 lines · Claudio Imbrenda
Previously, when a protected VM was rebooted or when it was shut down, its memory was made unprotected, and then the protected VM itself was destroyed. Looping over the whole address space can take some time, considering the overhead of the various Ultravisor Calls (UVCs). This means that a reboot or a shutdown would take a potentially long amount of time, depending on the amount of used memory.
Quoted 5 lines
This patchseries implements a deferred destroy mechanism for protected guests. When a protected guest is destroyed, its memory can be cleared in background, allowing the guest to restart or terminate significantly faster than before.
Patches 1-12 have spent a considerable amount of time in the CI and I'd like to queue them to be able to focus on the rest of the series.
Patch 9 will need two small fixups since there are two conflicts where a line was introduced before your addition of the include and the struct kvm_s390_pv mmu_notifier member. I.e. it's more of a patch history problem than a real conflict.
I'd fix that up when queuing if you're ok with it?
Claudio Imbrenda
maintainerQuoted 17 lines · On Thu, 14 Jul 2022 08:59:00 +0200 Janosch Frank <[email protected]> wrote:
Quoted 7 lines · Claudio Imbrenda
Previously, when a protected VM was rebooted or when it was shut down, its memory was made unprotected, and then the protected VM itself was destroyed. Looping over the whole address space can take some time, considering the overhead of the various Ultravisor Calls (UVCs). This means that a reboot or a shutdown would take a potentially long amount of time, depending on the amount of used memory.
Quoted 5 lines
This patchseries implements a deferred destroy mechanism for protected guests. When a protected guest is destroyed, its memory can be cleared in background, allowing the guest to restart or terminate significantly faster than before.
Patches 1-12 have spent a considerable amount of time in the CI and I'd like to queue them to be able to focus on the rest of the series.
Quoted 6 lines
Patch 9 will need two small fixups since there are two conflicts where a line was introduced before your addition of the include and the struct
kvm_s390_pvmmu_notifiermember. I.e. it's more of a patch history problem than a real conflict.I'd fix that up when queuing if you're ok with it?
ok for me