Migrate non-migrated pages of a SVM.
Subsystems (3)
v4 cover Migrate non-migrated pages of a SVM.
The time to switch a VM to Secure-VM, increases by the size of the VM. A 100GB VM takes about 7minutes. This is unacceptable. This linear increase is caused by a suboptimal behavior by the Ultravisor and the Hypervisor. The Ultravisor unnecessarily migrates all the GFN of the VM from normal-memory to secure-memory. It has to just migrate the necessary and sufficient GFNs.
However when the optimization is incorporated in the Ultravisor, the Hypervisor starts misbehaving. The Hypervisor has a inbuilt assumption that the Ultravisor will explicitly request to migrate, each and every GFN of the VM. If only necessary and sufficient GFNs are requested for migration, the Hypervisor continues to manage the remaining GFNs as normal GFNs. This leads to memory corruption; manifested consistently when the SVM reboots.
The same is true, when a memory slot is hotplugged into a SVM. The Hypervisor expects the ultravisor to request migration of all GFNs to secure-GFN. But the hypervisor cannot handle any H_SVM_PAGE_IN requests from the Ultravisor, done in the context of UV_REGISTER_MEM_SLOT ucall. This problem manifests as random errors in the SVM, when a memory-slot is hotplugged.
This patch series automatically migrates the non-migrated pages of a SVM, and thus solves the problem.
Testing: Passed rigorous testing using various sized SVMs.
Changelog:
v4: . Incorported Bharata's comments:
- Optimization -- replace write mmap semaphore with read mmap semphore.
- disable page-merge during memory hotplug.
- rearranged the patches. consolidated the page-migration-retry logic in a single patch.
v3: . Optimized the page-migration retry-logic.
. Relax and relinquish the cpu regularly while bulk migrating
the non-migrated pages. This issue was causing soft-lockups. Fixed it.
. Added a new patch, to retry page-migration a couple of times
before returning H_BUSY in H_SVM_PAGE_IN. This issue was seen a few times in a 24hour continuous reboot test of the SVMs.
v2: . fixed a bug observed by Laurent. The state of the GFN's associated
with Secure-VMs were not reset during memslot flush.
. Re-organized the code, for easier review.
. Better description of the patch series.
v1: fixed a bug observed by Bharata. Pages that where paged-in and later paged-out must also be skipped from migration during H_SVM_INIT_DONE.
Laurent Dufour (1):
KVM: PPC: Book3S HV: migrate hot plugged memory
Ram Pai (4):
KVM: PPC: Book3S HV: Disable page merging in H_SVM_INIT_START
KVM: PPC: Book3S HV: track the state GFNs associated with secure VMs
KVM: PPC: Book3S HV: in H_SVM_INIT_DONE, migrate remaining normal-GFNs
to secure-GFNs.
KVM: PPC: Book3S HV: retry page migration before erroring-out
Documentation/powerpc/ultravisor.rst | 4 + arch/powerpc/include/asm/kvm_book3s_uvmem.h | 12 + arch/powerpc/kvm/book3s_hv.c | 10 +- arch/powerpc/kvm/book3s_hv_uvmem.c | 508 ++++++++++++++++++++++++---- 4 files changed, 457 insertions(+), 77 deletions(-)
Signature · 1 line
1.8.3.1
PATCH v4 1/5 KVM: PPC: Book3S HV: Disable page merging in H_SVM_INIT_START
Page-merging of pages in memory-slots associated with a Secure VM, is disabled in H_SVM_PAGE_IN handler.
This operation should have been done much earlier; the moment the VM is initiated for secure-transition. Delaying this operation, increases the probability for those pages to acquire new references , making it impossible to migrate those pages.
Disable page-migration in H_SVM_INIT_START handling.
--- Documentation/powerpc/ultravisor.rst | 1 + arch/powerpc/kvm/book3s_hv_uvmem.c | 98 +++++++++++++++++++++++++++--------- 2 files changed, 76 insertions(+), 23 deletions(-)
Documentation/powerpc/ultravisor.rst +1 -0
@@ -895,6 +895,7 @@ Return values One of the following values: * H_SUCCESS on success. + * H_STATE if the VM is not in a position to switch to secure. Description ~~~~~~~~~~~
arch/powerpc/kvm/book3s_hv_uvmem.c +75 -23
156 lines not on this page · raw message
Signature · 1 line
1.8.3.1
Re: [v4 1/5] KVM: PPC: Book3S HV: Disable page merging in H_SVM_INIT_START
Quoted 11 lines · Ram Pai
Page-merging of pages in memory-slots associated with a Secure VM, is disabled in
H_SVM_PAGE_INhandler.This operation should have been done much earlier; the moment the VM is initiated for secure-transition. Delaying this operation, increases the probability for those pages to acquire new references , making it impossible to migrate those pages.
Disable page-migration in
H_SVM_INIT_STARThandling.Signed-off-by Ram Pai <[email protected]>
with a few observations below...
Quoted 43 lines
--- Documentation/powerpc/ultravisor.rst | 1 + arch/powerpc/kvm/book3s_hv_uvmem.c | 98 +++++++++++++++++++++++++++--------- 2 files changed, 76 insertions(+), 23 deletions(-)Documentation/powerpc/ultravisor.rst +1
-0@@ -895,6 +895,7 @@ Return values One of the following values: * H_SUCCESS on success. + * H_STATE if the VM is not in a position to switch to secure. Description ~~~~~~~~~~~arch/powerpc/kvm/book3s_hv_uvmem.c +18
-0@@ -211,6 +211,65 @@ static bool kvmppc_gfn_is_uvmem_pfn(unsigned long gfn, struct kvm *kvm,return false; } +static int kvmppc_memslot_page_merge (struct kvm *kvm, +struct kvm_memory_slot *memslot,bool merge) +{ +unsigned long gfn = memslot->base_gfn; +unsigned long end, start =gfn_to_hva (kvm, gfn); +int ret =0 ; +struct vm_area_struct *vma; +int merge_flag = (merge) ?MADV_MERGEABLE :MADV_UNMERGEABLE ; + +if (kvm_is_error_hva (start)) +return H_STATE ; + + end = start + (memslot->npages <<PAGE_SHIFT ); + +mmap_write_lock (kvm->mm); + do { + vma = find_vma_intersection(kvm->mm, start, end); + if (!vma) {
arch/powerpc/kvm/book3s_hv_uvmem.c static bool kvmppc_gfn_is_uvmem_pfn(unsigned long gfn, struct kvm *kvm, · line 241
Quoted 10 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +10
-0+ ret =H_STATE ; +break ; + } + ret =ksm_madvise (vma, vma->vm_start, vma->vm_end, + merge_flag, &vma->vm_flags); +if (ret) { + ret =H_STATE ; +break ; + } + start = vma->vm_end +1 ;
This should be start = vma->vm_end I believe.
Quoted 10 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +10
-0+ }while (end > vma->vm_end); + +mmap_write_unlock (kvm->mm); +return ret; +} + +static int __kvmppc_page_merge (struct kvm *kvm,bool merge) +{ +struct kvm_memslots *slots; +struct kvm_memory_slot *memslot;
arch/powerpc/kvm/book3s_hv_uvmem.c static bool kvmppc_gfn_is_uvmem_pfn(unsigned long gfn, struct kvm *kvm, · line 261
Quoted 10 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +10
-0+int ret =0 ; + + slots =kvm_memslots (kvm); +kvm_for_each_memslot (memslot, slots) { + ret =kvmppc_memslot_page_merge (kvm, memslot, merge); +if (ret) +break ; + } +return ret; +}
You walk through all the slots here to issue kvm_madvise, but...
Quoted 17 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +11
-0+ +static inline int kvmppc_disable_page_merge (struct kvm *kvm) +{ +return __kvmppc_page_merge (kvm, false); +} + +static inline int kvmppc_enable_page_merge (struct kvm *kvm) +{ +return __kvmppc_page_merge (kvm, true); +} +unsigned long kvmppc_h_svm_init_start (struct kvm *kvm) {struct kvm_memslots *slots;@@ -232,11 +291,18 @@ unsigned long kvmppc_h_svm_init_start(struct kvm *kvm)return H_AUTHORITY ;
arch/powerpc/kvm/book3s_hv_uvmem.c unsigned long kvmppc_h_svm_init_start(struct kvm *kvm) · line 302
Quoted 10 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +7
-0srcu_idx =srcu_read_lock (&kvm->srcu); + +/* disable page-merging for all memslot */ + ret =kvmppc_disable_page_merge (kvm); +if (ret) + goto out; + +/* register the memslot */ slots =kvm_memslots (kvm);kvm_for_each_memslot (memslot, slots) {
... you are walking thro' the same set of slots here anyway. I think it makes sense to issue merge advices from here itself. That will help you to share code with kvmppc_memslot_create() in 5/5.
All the below 3 calls are common to both the code paths, I think they can be carved out into a separate function if you prefer.
kvmppc_uvmem_slot_initkvmppc_memslot_page_mergeuv_register_mem_slot
Regards,
Bharata.
PATCH v4 2/5 KVM: PPC: Book3S HV: track the state GFNs associated with secure VMs
In reply to Ram PaiDuring the life of SVM, its GFNs transition through normal, secure and shared states. Since the kernel does not track GFNs that are shared, it is not possible to disambiguate a shared GFN from a GFN whose PFN has not yet been migrated to a secure-PFN. Also it is not possible to disambiguate a secure-GFN from a GFN whose GFN has been pagedout from the ultravisor.
The ability to identify the state of a GFN is needed to skip migration of its PFN to secure-PFN during ESM transition.
The code is re-organized to track the states of a GFN as explained below.
************************************************************************
1. States of a GFN
---------------
The GFN can be in one of the following states.
(a) Secure - The GFN is secure. The GFN is associated with a Secure VM, the contents of the GFN is not accessible to the Hypervisor. This GFN can be backed by a secure-PFN, or can be backed by a normal-PFN with contents encrypted. The former is true when the GFN is paged-in into the ultravisor. The latter is true when the GFN is paged-out of the ultravisor.
(b) Shared - The GFN is shared. The GFN is associated with a a secure VM. The contents of the GFN is accessible to Hypervisor. This GFN is backed by a normal-PFN and its content is un-encrypted.
(c) Normal - The GFN is a normal. The GFN is associated with a normal VM. The contents of the GFN is accesible to the Hypervisor. Its content is never encrypted.
2. States of a VM.
---------------
(a) Normal VM: A VM whose contents are always accessible to the hypervisor. All its GFNs are normal-GFNs.
(b) Secure VM: A VM whose contents are not accessible to the hypervisor without the VM's consent. Its GFNs are either Shared-GFN or Secure-GFNs.
(c) Transient VM: A Normal VM that is transitioning to secure VM. The transition starts on successful return of H_SVM_INIT_START, and ends on successful return of H_SVM_INIT_DONE. This transient VM, can have GFNs in any of the three states; i.e Secure-GFN, Shared-GFN,
and Normal-GFN. The VM never executes in this state
in supervisor-mode.
3. Memory slot State.
------------------
The state of a memory slot mirrors the state of the VM the memory slot is associated with.
4. VM State transition.
--------------------
A VM always starts in Normal Mode.
H_SVM_INIT_START moves the VM into transient state. During this time the Ultravisor may request some of its GFNs to be shared or secured. So its GFNs can be in one of the three GFN states.
H_SVM_INIT_DONE moves the VM entirely from transient state to secure-state. At this point any left-over normal-GFNs are transitioned to Secure-GFN.
H_SVM_INIT_ABORT moves the transient VM back to normal VM. All its GFNs are moved to Normal-GFNs.
UV_TERMINATE transitions the secure-VM back to normal-VM. All the secure-GFN and shared-GFNs are tranistioned to normal-GFN
Note: The contents of the normal-GFN is undefined at this point.
5. GFN state implementation:
-------------------------
Secure GFN is associated with a secure-PFN; also called uvmem_pfn, when the GFN is paged-in. Its pfn[] has KVMPPC_GFN_UVMEM_PFN flag set, and contains the value of the secure-PFN.
It is associated with a normal-PFN; also called mem_pfn, when the GFN is pagedout. Its pfn[] has KVMPPC_GFN_MEM_PFN flag set. The value of the normal-PFN is not tracked.
Shared GFN is associated with a normal-PFN. Its pfn[] has KVMPPC_UVMEM_SHARED_PFN flag set. The value of the normal-PFN is not tracked.
Normal GFN is associated with normal-PFN. Its pfn[] has no flag set. The value of the normal-PFN is not tracked.
6. Life cycle of a GFN
-------------------- -------------------------------------------------------------- | | Share | Unshare | SVM |H_SVM_INIT_DONE| | |operation |operation | abort/ | | | | | | terminate | | ------------------------------------------------------------- | | | | | | | Secure | Shared | Secure |Normal |Secure | | | | | | | | Shared | Shared | Secure |Normal |Shared | | | | | | | | Normal | Shared | Secure |Normal |Secure |
--------------------------------------------------------------
7. Life cycle of a VM
-------------------- -------------------------------------------------------------------- | | start | H_SVM_ |H_SVM_ |H_SVM_ |UV_SVM_ | | | VM |INIT_START|INIT_DONE|INIT_ABORT |TERMINATE | | | | | | | | --------- ---------------------------------------------------------- | | | | | | | | Normal | Normal | Transient|Error |Error |Normal | | | | | | | | | Secure | Error | Error |Error |Error |Normal | | | | | | | | |Transient| N/A | Error |Secure |Normal |Normal |
--------------------------------------------------------------------
************************************************************************
--- arch/powerpc/kvm/book3s_hv_uvmem.c | 187 +++++++++++++++++++++++++++++++++---- 1 file changed, 168 insertions(+), 19 deletions(-)
arch/powerpc/kvm/book3s_hv_uvmem.c +168 -19
@@ -98,7 +98,127 @@static unsigned long *kvmppc_uvmem_bitmap;static DEFINE_SPINLOCK (kvmppc_uvmem_bitmap_lock); -#define KVMPPC_UVMEM_PFN (1UL << 63) +/* +* States of a GFN +* --------------- +* The GFN can be in one of the following states. +* +* (a) Secure - The GFN is secure. The GFN is associated with +* a Secure VM, the contents of the GFN is not accessible +* to the Hypervisor. This GFN can be backed by a secure-PFN, +* or can be backed by a normal-PFN with contents encrypted. +* The former is true when the GFN is paged-in into the +* ultravisor. The latter is true when the GFN is paged-out +* of the ultravisor. +* +* (b) Shared - The GFN is shared. The GFN is associated with a +* a secure VM. The contents of the GFN is accessible to +* Hypervisor. This GFN is backed by a normal-PFN and its +* content is un-encrypted. +* +* (c) Normal - The GFN is a normal. The GFN is associated with +* a normal VM. The contents of the GFN is accesible to +* the Hypervisor. Its content is never encrypted. +* +* States of a VM. +* --------------- +* +* Normal VM: A VM whose contents are always accessible to +* the hypervisor. All its GFNs are normal-GFNs. +* +* Secure VM: A VM whose contents are not accessible to the +* hypervisor without the VM's consent. Its GFNs are +* either Shared-GFN or Secure-GFNs. +* +* Transient VM: A Normal VM that is transitioning to secure VM. +* The transition starts on successful return of +* H_SVM_INIT_START, and ends on successful return +* of H_SVM_INIT_DONE. This transient VM, can have GFNs +* in any of the three states; i.e Secure-GFN, Shared-GFN, +* and Normal-GFN. The VM never executes in this state +* in supervisor-mode. +* +* Memory slot State. +* ----------------------------- +* The state of a memory slot mirrors the state of the +* VM the memory slot is associated with. +* +* VM State transition. +* -------------------- +* +* A VM always starts in Normal Mode. +* +* H_SVM_INIT_START moves the VM into transient state. During this +* time the Ultravisor may request some of its GFNs to be shared or +* secured. So its GFNs can be in one of the three GFN states. +* +* H_SVM_INIT_DONE moves the VM entirely from transient state to +* secure-state. At this point any left-over normal-GFNs are +* transitioned to Secure-GFN. +* +* H_SVM_INIT_ABORT moves the transient VM back to normal VM. +* All its GFNs are moved to Normal-GFNs. +* +* UV_TERMINATE transitions the secure-VM back to normal-VM. All +* the secure-GFN and shared-GFNs are tranistioned to normal-GFN +* Note: The contents of the normal-GFN is undefined at this point. +* +* GFN state implementation: +* ------------------------- +* +* Secure GFN is associated with a secure-PFN; also called uvmem_pfn, +* when the GFN is paged-in. Its pfn[] has KVMPPC_GFN_UVMEM_PFN flag +* set, and contains the value of the secure-PFN. +* It is associated with a normal-PFN; also called mem_pfn, when +* the GFN is pagedout. Its pfn[] has KVMPPC_GFN_MEM_PFN flag set. +* The value of the normal-PFN is not tracked. +* +* Shared GFN is associated with a normal-PFN. Its pfn[] has +* KVMPPC_UVMEM_SHARED_PFN flag set. The value of the normal-PFN +* is not tracked. +* +* Normal GFN is associated with normal-PFN. Its pfn[] has +* no flag set. The value of the normal-PFN is not tracked. +* +* Life cycle of a GFN +* -------------------- +* +* -------------------------------------------------------------- +* | | Share | Unshare | SVM |H_SVM_INIT_DONE| +* | |operation |operation | abort/ | | +* | | | | terminate | | +* ------------------------------------------------------------- +* | | | | | | +* | Secure | Shared | Secure |Normal |Secure | +* | | | | | | +* | Shared | Shared | Secure |Normal |Shared | +* | | | | | | +* | Normal | Shared | Secure |Normal |Secure | +* -------------------------------------------------------------- +* +* Life cycle of a VM +* -------------------- +* +* -------------------------------------------------------------------- +* | | start | H_SVM_ |H_SVM_ |H_SVM_ |UV_SVM_ | +* | | VM |INIT_START|INIT_DONE|INIT_ABORT |TERMINATE | +* | | | | | | | +* --------- ---------------------------------------------------------- +* | | | | | | | +* | Normal | Normal | Transient|Error |Error |Normal | +* | | | | | | | +* | Secure | Error | Error |Error |Error |Normal | +* | | | | | | | +* |Transient| N/A | Error |Secure |Normal |Normal | +* -------------------------------------------------------------------- +*/ + +#define KVMPPC_GFN_UVMEM_PFN (1UL << 63) +#define KVMPPC_GFN_MEM_PFN (1UL << 62) +#define KVMPPC_GFN_SHARED (1UL << 61) +#define KVMPPC_GFN_SECURE (KVMPPC_GFN_UVMEM_PFN | KVMPPC_GFN_MEM_PFN) +#define KVMPPC_GFN_FLAG_MASK (KVMPPC_GFN_SECURE | KVMPPC_GFN_SHARED) +#define KVMPPC_GFN_PFN_MASK (~KVMPPC_GFN_FLAG_MASK)struct kvmppc_uvmem_slot {struct list_head list;
@@ -106,11 +226,11 @@ struct kvmppc_uvmem_slot {unsigned long base_pfn;unsigned long *pfns; }; -struct kvmppc_uvmem_page_pvt {struct kvm *kvm;unsigned long gpa;bool skip_page_out; +bool remove_gfn; };bool kvmppc_uvmem_available (void )
@@ -163,8 +283,8 @@ void kvmppc_uvmem_slot_free(struct kvm *kvm, const struct kvm_memory_slot *slot)mutex_unlock (&kvm->arch.uvmem_lock); } -static void kvmppc_uvmem_pfn_insert (unsigned long gfn,unsigned long uvmem_pfn, -struct kvm *kvm) +static void kvmppc_mark_gfn (unsigned long gfn,struct kvm *kvm, +unsigned long flag,unsigned long uvmem_pfn) {struct kvmppc_uvmem_slot *p;
@@ -172,24 +292,41 @@ static void kvmppc_uvmem_pfn_insert(unsigned long gfn, unsigned long uvmem_pfn,if (gfn >= p->base_pfn && gfn < p->base_pfn + p->nr_pfns) {unsigned long index = gfn - p->base_pfn; - p->pfns[index] = uvmem_pfn |KVMPPC_UVMEM_PFN ; +if (flag =KVMPPC_GFN_UVMEM_PFN ) + p->pfns[index] = uvmem_pfn | flag; +else + p->pfns[index] = flag;return ; } } } -static void kvmppc_uvmem_pfn_remove (unsigned long gfn,struct kvm *kvm) +/* mark the GFN as secure-GFN associated with @uvmem pfn device-PFN. */ +static void kvmppc_gfn_secure_uvmem_pfn (unsigned long gfn, +unsigned long uvmem_pfn,struct kvm *kvm) { -struct kvmppc_uvmem_slot *p; +kvmppc_mark_gfn (gfn, kvm,KVMPPC_GFN_UVMEM_PFN , uvmem_pfn); +} -list_for_each_entry (p, &kvm->arch.uvmem_pfns, list) { -if (gfn >= p->base_pfn && gfn < p->base_pfn + p->nr_pfns) { - p->pfns[gfn - p->base_pfn] =0 ; -return ; - } - } +/* mark the GFN as secure-GFN associated with a memory-PFN. */ +static void kvmppc_gfn_secure_mem_pfn (unsigned long gfn,struct kvm *kvm) +{ +kvmppc_mark_gfn (gfn, kvm,KVMPPC_GFN_MEM_PFN ,0 ); +} + +/* mark the GFN as a shared GFN. */ +static void kvmppc_gfn_shared (unsigned long gfn,struct kvm *kvm) +{ +kvmppc_mark_gfn (gfn, kvm,KVMPPC_GFN_SHARED ,0 ); +} + +/* mark the GFN as a non-existent GFN. */ +static void kvmppc_gfn_remove (unsigned long gfn,struct kvm *kvm) +{ +kvmppc_mark_gfn (gfn, kvm,0 ,0 ); } +/* return true, if the GFN is a secure-GFN backed by a secure-PFN */ static bool kvmppc_gfn_is_uvmem_pfn (unsigned long gfn,struct kvm *kvm,unsigned long *uvmem_pfn) {
@@ -199,10 +336,10 @@ static bool kvmppc_gfn_is_uvmem_pfn(unsigned long gfn, struct kvm *kvm,if (gfn >= p->base_pfn && gfn < p->base_pfn + p->nr_pfns) {unsigned long index = gfn - p->base_pfn; -if (p->pfns[index] &KVMPPC_UVMEM_PFN ) { +if (p->pfns[index] &KVMPPC_GFN_UVMEM_PFN ) {if (uvmem_pfn) *uvmem_pfn = p->pfns[index] & - ~KVMPPC_UVMEM_PFN ; +KVMPPC_GFN_PFN_MASK ;return true; }else return false;
@@ -353,6 +490,7 @@ void kvmppc_uvmem_drop_pages(const struct kvm_memory_slot *free,mutex_lock (&kvm->arch.uvmem_lock);if (!kvmppc_gfn_is_uvmem_pfn (gfn, kvm, &uvmem_pfn)) { +kvmppc_gfn_remove (gfn, kvm);mutex_unlock (&kvm->arch.uvmem_lock);continue ; }
@@ -360,6 +498,7 @@ void kvmppc_uvmem_drop_pages(const struct kvm_memory_slot *free, uvmem_page =pfn_to_page (uvmem_pfn); pvt = uvmem_page->zone_device_data; pvt->skip_page_out = skip_page_out; + pvt->remove_gfn = true;mutex_unlock (&kvm->arch.uvmem_lock); pfn =gfn_to_pfn (kvm, gfn);
@@ -429,7 +568,7 @@ static struct page *kvmppc_uvmem_get_page(unsigned long gpa, struct kvm *kvm) goto out_clear; uvmem_pfn = bit + pfn_first; -kvmppc_uvmem_pfn_insert (gpa >>PAGE_SHIFT , uvmem_pfn, kvm); +kvmppc_gfn_secure_uvmem_pfn (gpa >>PAGE_SHIFT , uvmem_pfn, kvm); pvt->gpa = gpa; pvt->kvm = kvm;
@@ -524,6 +663,7 @@ static unsigned long kvmppc_share_page(struct kvm *kvm, unsigned long gpa, uvmem_page =pfn_to_page (uvmem_pfn); pvt = uvmem_page->zone_device_data; pvt->skip_page_out = true; + pvt->remove_gfn = false; } retry:
@@ -537,12 +677,16 @@ static unsigned long kvmppc_share_page(struct kvm *kvm, unsigned long gpa, uvmem_page =pfn_to_page (uvmem_pfn); pvt = uvmem_page->zone_device_data; pvt->skip_page_out = true; + pvt->remove_gfn = false;kvm_release_pfn_clean (pfn); goto retry; } -if (!uv_page_in (kvm->arch.lpid, pfn << page_shift, gpa,0 , page_shift)) +if (!uv_page_in (kvm->arch.lpid, pfn << page_shift, gpa,0 , + page_shift)) { +kvmppc_gfn_shared (gfn, kvm); ret =H_SUCCESS ; + }kvm_release_pfn_clean (pfn);mutex_unlock (&kvm->arch.uvmem_lock); out:
@@ -598,6 +742,7 @@ unsigned long kvmppc_h_svm_page_in(struct kvm *kvm, unsigned long gpa,if (!kvmppc_svm_page_in (vma, start, end, gpa, kvm, page_shift)) ret =H_SUCCESS ; + out_unlock:mutex_unlock (&kvm->arch.uvmem_lock); out:
@@ -706,7 +851,8 @@ static vm_fault_t kvmppc_uvmem_migrate_to_ram(struct vm_fault *vmf)/* * Release the device PFN back to the pool * -* Gets called when secure page becomes a normal page during H_SVM_PAGE_OUT. +* Gets called when secure GFN tranistions from a secure-PFN +* to a normal PFN during H_SVM_PAGE_OUT. * Gets called with kvm->arch.uvmem_lock held. */ static void kvmppc_uvmem_page_free (struct page *page)
@@ -721,7 +867,10 @@ static void kvmppc_uvmem_page_free(struct page *page) pvt = page->zone_device_data; page->zone_device_data =NULL ; -kvmppc_uvmem_pfn_remove (pvt->gpa >>PAGE_SHIFT , pvt->kvm); +if (pvt->remove_gfn) +kvmppc_gfn_remove (pvt->gpa >>PAGE_SHIFT , pvt->kvm); +else +kvmppc_gfn_secure_mem_pfn (pvt->gpa >>PAGE_SHIFT , pvt->kvm);kfree (pvt); }
Signature · 1 line
1.8.3.1
Re: [v4 2/5] KVM: PPC: Book3S HV: track the state GFNs associated with secure VMs
Quoted 400 lines · Ram Pai
During the life of SVM, its GFNs transition through normal, secure and shared states. Since the kernel does not track GFNs that are shared, it is not possible to disambiguate a shared GFN from a GFN whose PFN has not yet been migrated to a secure-PFN. Also it is not possible to disambiguate a secure-GFN from a GFN whose GFN has been pagedout from the ultravisor.
The ability to identify the state of a GFN is needed to skip migration of its PFN to secure-PFN during ESM transition.
The code is re-organized to track the states of a GFN as explained below.
************************************************************************
1. States of a GFN---------------The GFN can be in one of the following states.
(a) Secure - The GFN is secure. The GFN is associated with
(quoted diff) +0
-0a Secure VM, the contents of the GFN is not accessible to the Hypervisor. This GFN can be backed by a secure-PFN, or can be backed by a normal-PFN with contents encrypted. The former is true when the GFN is paged-in into the ultravisor. The latter is true when the GFN is paged-out of the ultravisor. (b) Shared - The GFN is shared. The GFN is associated with a a secure VM. The contents of the GFN is accessible to Hypervisor. This GFN is backed by a normal-PFN and its content is un-encrypted. (c) Normal - The GFN is a normal. The GFN is associated with a normal VM. The contents of the GFN is accesible to the Hypervisor. Its content is never encrypted. 2. States of a VM. --------------- (a) Normal VM: A VM whose contents are always accessible to the hypervisor. All its GFNs are normal-GFNs. (b) Secure VM: A VM whose contents are not accessible to the hypervisor without the VM's consent. Its GFNs are either Shared-GFN or Secure-GFNs. (c) Transient VM: A Normal VM that is transitioning to secure VM. The transition starts on successful return of H_SVM_INIT_START, and ends on successful return of H_SVM_INIT_DONE. This transient VM, can have GFNs in any of the three states; i.e Secure-GFN, Shared-GFN, and Normal-GFN. The VM never executes in this state in supervisor-mode. 3. Memory slot State. ------------------ The state of a memory slot mirrors the state of the VM the memory slot is associated with. 4. VM State transition. -------------------- A VM always starts in Normal Mode. H_SVM_INIT_START moves the VM into transient state. During this time the Ultravisor may request some of its GFNs to be shared or secured. So its GFNs can be in one of the three GFN states. H_SVM_INIT_DONE moves the VM entirely from transient state to secure-state. At this point any left-over normal-GFNs are transitioned to Secure-GFN. H_SVM_INIT_ABORT moves the transient VM back to normal VM. All its GFNs are moved to Normal-GFNs. UV_TERMINATE transitions the secure-VM back to normal-VM. All the secure-GFN and shared-GFNs are tranistioned to normal-GFN Note: The contents of the normal-GFN is undefined at this point. 5. GFN state implementation: ------------------------- Secure GFN is associated with a secure-PFN; also called uvmem_pfn, when the GFN is paged-in. Its pfn[] has KVMPPC_GFN_UVMEM_PFN flag set, and contains the value of the secure-PFN. It is associated with a normal-PFN; also called mem_pfn, when the GFN is pagedout. Its pfn[] has KVMPPC_GFN_MEM_PFN flag set. The value of the normal-PFN is not tracked. Shared GFN is associated with a normal-PFN. Its pfn[] has KVMPPC_UVMEM_SHARED_PFN flag set. The value of the normal-PFN is not tracked. Normal GFN is associated with normal-PFN. Its pfn[] has no flag set. The value of the normal-PFN is not tracked. 6. Life cycle of a GFN -------------------- -------------------------------------------------------------- | | Share | Unshare | SVM |H_SVM_INIT_DONE| | |operation |operation | abort/ | | | | | | terminate | | ------------------------------------------------------------- | | | | | | | Secure | Shared | Secure |Normal |Secure | | | | | | | | Shared | Shared | Secure |Normal |Shared | | | | | | | | Normal | Shared | Secure |Normal |Secure | -------------------------------------------------------------- 7. Life cycle of a VM -------------------- -------------------------------------------------------------------- | | start | H_SVM_ |H_SVM_ |H_SVM_ |UV_SVM_ | | | VM |INIT_START|INIT_DONE|INIT_ABORT |TERMINATE | | | | | | | | --------- ---------------------------------------------------------- | | | | | | | | Normal | Normal | Transient|Error |Error |Normal | | | | | | | | | Secure | Error | Error |Error |Error |Normal | | | | | | | | |Transient| N/A | Error |Secure |Normal |Normal | --------------------------------------------------------------------************************************************************************
Cc Paul Mackerras <[email protected]>Cc Benjamin Herrenschmidt <[email protected]>Cc Michael Ellerman <[email protected]>Cc Bharata B Rao <[email protected]>Cc Aneesh Kumar K.V <[email protected]>Cc Sukadev Bhattiprolu <[email protected]>Cc Laurent Dufour <[email protected]>Cc Thiago Jung Bauermann <[email protected]>Cc David Gibson <[email protected]>Cc Claudio Carvalho <[email protected]>Reviewed-by Thiago Jung Bauermann <[email protected]>Signed-off-by Ram Pai <[email protected]>--- arch/powerpc/kvm/book3s_hv_uvmem.c | 187 +++++++++++++++++++++++++++++++++---- 1 file changed, 168 insertions(+), 19 deletions(-)arch/powerpc/kvm/book3s_hv_uvmem.c +156
-16@@ -98,7 +98,127 @@static unsigned long *kvmppc_uvmem_bitmap;static DEFINE_SPINLOCK (kvmppc_uvmem_bitmap_lock); -#define KVMPPC_UVMEM_PFN (1UL << 63) +/* +* States of a GFN +* --------------- +* The GFN can be in one of the following states. +* +* (a) Secure - The GFN is secure. The GFN is associated with +* a Secure VM, the contents of the GFN is not accessible +* to the Hypervisor. This GFN can be backed by a secure-PFN, +* or can be backed by a normal-PFN with contents encrypted. +* The former is true when the GFN is paged-in into the +* ultravisor. The latter is true when the GFN is paged-out +* of the ultravisor. +* +* (b) Shared - The GFN is shared. The GFN is associated with a +* a secure VM. The contents of the GFN is accessible to +* Hypervisor. This GFN is backed by a normal-PFN and its +* content is un-encrypted. +* +* (c) Normal - The GFN is a normal. The GFN is associated with +* a normal VM. The contents of the GFN is accesible to +* the Hypervisor. Its content is never encrypted. +* +* States of a VM. +* --------------- +* +* Normal VM: A VM whose contents are always accessible to +* the hypervisor. All its GFNs are normal-GFNs. +* +* Secure VM: A VM whose contents are not accessible to the +* hypervisor without the VM's consent. Its GFNs are +* either Shared-GFN or Secure-GFNs. +* +* Transient VM: A Normal VM that is transitioning to secure VM. +* The transition starts on successful return of +* H_SVM_INIT_START, and ends on successful return +* of H_SVM_INIT_DONE. This transient VM, can have GFNs +* in any of the three states; i.e Secure-GFN, Shared-GFN, +* and Normal-GFN. The VM never executes in this state +* in supervisor-mode. +* +* Memory slot State. +* ----------------------------- +* The state of a memory slot mirrors the state of the +* VM the memory slot is associated with. +* +* VM State transition. +* -------------------- +* +* A VM always starts in Normal Mode. +* +* H_SVM_INIT_START moves the VM into transient state. During this +* time the Ultravisor may request some of its GFNs to be shared or +* secured. So its GFNs can be in one of the three GFN states. +* +* H_SVM_INIT_DONE moves the VM entirely from transient state to +* secure-state. At this point any left-over normal-GFNs are +* transitioned to Secure-GFN. +* +* H_SVM_INIT_ABORT moves the transient VM back to normal VM. +* All its GFNs are moved to Normal-GFNs. +* +* UV_TERMINATE transitions the secure-VM back to normal-VM. All +* the secure-GFN and shared-GFNs are tranistioned to normal-GFN +* Note: The contents of the normal-GFN is undefined at this point. +* +* GFN state implementation: +* ------------------------- +* +* Secure GFN is associated with a secure-PFN; also called uvmem_pfn, +* when the GFN is paged-in. Its pfn[] has KVMPPC_GFN_UVMEM_PFN flag +* set, and contains the value of the secure-PFN. +* It is associated with a normal-PFN; also called mem_pfn, when +* the GFN is pagedout. Its pfn[] has KVMPPC_GFN_MEM_PFN flag set. +* The value of the normal-PFN is not tracked. +* +* Shared GFN is associated with a normal-PFN. Its pfn[] has +* KVMPPC_UVMEM_SHARED_PFN flag set. The value of the normal-PFN +* is not tracked. +* +* Normal GFN is associated with normal-PFN. Its pfn[] has +* no flag set. The value of the normal-PFN is not tracked. +* +* Life cycle of a GFN +* -------------------- +* +* -------------------------------------------------------------- +* | | Share | Unshare | SVM |H_SVM_INIT_DONE| +* | |operation |operation | abort/ | | +* | | | | terminate | | +* ------------------------------------------------------------- +* | | | | | | +* | Secure | Shared | Secure |Normal |Secure | +* | | | | | | +* | Shared | Shared | Secure |Normal |Shared | +* | | | | | | +* | Normal | Shared | Secure |Normal |Secure | +* -------------------------------------------------------------- +* +* Life cycle of a VM +* -------------------- +* +* -------------------------------------------------------------------- +* | | start | H_SVM_ |H_SVM_ |H_SVM_ |UV_SVM_ | +* | | VM |INIT_START|INIT_DONE|INIT_ABORT |TERMINATE | +* | | | | | | | +* --------- ---------------------------------------------------------- +* | | | | | | | +* | Normal | Normal | Transient|Error |Error |Normal | +* | | | | | | | +* | Secure | Error | Error |Error |Error |Normal | +* | | | | | | | +* |Transient| N/A | Error |Secure |Normal |Normal | +* -------------------------------------------------------------------- +*/ + +#define KVMPPC_GFN_UVMEM_PFN (1UL << 63) +#define KVMPPC_GFN_MEM_PFN (1UL << 62) +#define KVMPPC_GFN_SHARED (1UL << 61) +#define KVMPPC_GFN_SECURE (KVMPPC_GFN_UVMEM_PFN | KVMPPC_GFN_MEM_PFN) +#define KVMPPC_GFN_FLAG_MASK (KVMPPC_GFN_SECURE | KVMPPC_GFN_SHARED) +#define KVMPPC_GFN_PFN_MASK (~KVMPPC_GFN_FLAG_MASK)struct kvmppc_uvmem_slot {struct list_head list;@@ -106,11 +226,11 @@ struct kvmppc_uvmem_slot {unsigned long base_pfn;unsigned long *pfns; }; -struct kvmppc_uvmem_page_pvt {struct kvm *kvm;unsigned long gpa;bool skip_page_out; +bool remove_gfn; };bool kvmppc_uvmem_available (void )@@ -163,8 +283,8 @@ void kvmppc_uvmem_slot_free(struct kvm *kvm, const struct kvm_memory_slot *slot)mutex_unlock (&kvm->arch.uvmem_lock); } -static void kvmppc_uvmem_pfn_insert (unsigned long gfn,unsigned long uvmem_pfn, -struct kvm *kvm) +static void kvmppc_mark_gfn (unsigned long gfn,struct kvm *kvm, +unsigned long flag,unsigned long uvmem_pfn) {struct kvmppc_uvmem_slot *p;@@ -172,24 +292,41 @@ static void kvmppc_uvmem_pfn_insert(unsigned long gfn, unsigned long uvmem_pfn,if (gfn >= p->base_pfn && gfn < p->base_pfn + p->nr_pfns) {unsigned long index = gfn - p->base_pfn; - p->pfns[index] = uvmem_pfn |KVMPPC_UVMEM_PFN ; +if (flag =KVMPPC_GFN_UVMEM_PFN ) + p->pfns[index] = uvmem_pfn | flag; +else + p->pfns[index] = flag;return ; } } } -static void kvmppc_uvmem_pfn_remove (unsigned long gfn,struct kvm *kvm) +/* mark the GFN as secure-GFN associated with @uvmem pfn device-PFN. */ +static void kvmppc_gfn_secure_uvmem_pfn (unsigned long gfn, +unsigned long uvmem_pfn,struct kvm *kvm) { -struct kvmppc_uvmem_slot *p; +kvmppc_mark_gfn (gfn, kvm,KVMPPC_GFN_UVMEM_PFN , uvmem_pfn); +} -list_for_each_entry (p, &kvm->arch.uvmem_pfns, list) { -if (gfn >= p->base_pfn && gfn < p->base_pfn + p->nr_pfns) { - p->pfns[gfn - p->base_pfn] =0 ; -return ; - } - } +/* mark the GFN as secure-GFN associated with a memory-PFN. */ +static void kvmppc_gfn_secure_mem_pfn (unsigned long gfn,struct kvm *kvm) +{ +kvmppc_mark_gfn (gfn, kvm,KVMPPC_GFN_MEM_PFN ,0 ); +} + +/* mark the GFN as a shared GFN. */ +static void kvmppc_gfn_shared (unsigned long gfn,struct kvm *kvm) +{ +kvmppc_mark_gfn (gfn, kvm,KVMPPC_GFN_SHARED ,0 ); +} + +/* mark the GFN as a non-existent GFN. */ +static void kvmppc_gfn_remove (unsigned long gfn,struct kvm *kvm) +{ +kvmppc_mark_gfn (gfn, kvm,0 ,0 ); } +/* return true, if the GFN is a secure-GFN backed by a secure-PFN */ static bool kvmppc_gfn_is_uvmem_pfn (unsigned long gfn,struct kvm *kvm,unsigned long *uvmem_pfn) {@@ -199,10 +336,10 @@ static bool kvmppc_gfn_is_uvmem_pfn(unsigned long gfn, struct kvm *kvm,if (gfn >= p->base_pfn && gfn < p->base_pfn + p->nr_pfns) {unsigned long index = gfn - p->base_pfn; -if (p->pfns[index] &KVMPPC_UVMEM_PFN ) { +if (p->pfns[index] &KVMPPC_GFN_UVMEM_PFN ) {if (uvmem_pfn) *uvmem_pfn = p->pfns[index] & - ~KVMPPC_UVMEM_PFN ; +KVMPPC_GFN_PFN_MASK ;return true; }else return false;@@ -353,6 +490,7 @@ void kvmppc_uvmem_drop_pages(const struct kvm_memory_slot *free,mutex_lock (&kvm->arch.uvmem_lock);if (!kvmppc_gfn_is_uvmem_pfn (gfn, kvm, &uvmem_pfn)) { +kvmppc_gfn_remove (gfn, kvm);mutex_unlock (&kvm->arch.uvmem_lock);continue ; }@@ -360,6 +498,7 @@ void kvmppc_uvmem_drop_pages(const struct kvm_memory_slot *free, uvmem_page =pfn_to_page (uvmem_pfn); pvt = uvmem_page->zone_device_data; pvt->skip_page_out = skip_page_out; + pvt->remove_gfn = true;mutex_unlock (&kvm->arch.uvmem_lock); pfn =gfn_to_pfn (kvm, gfn);@@ -429,7 +568,7 @@ static struct page *kvmppc_uvmem_get_page(unsigned long gpa, struct kvm *kvm) goto out_clear; uvmem_pfn = bit + pfn_first; -kvmppc_uvmem_pfn_insert (gpa >>PAGE_SHIFT , uvmem_pfn, kvm); +kvmppc_gfn_secure_uvmem_pfn (gpa >>PAGE_SHIFT , uvmem_pfn, kvm); pvt->gpa = gpa; pvt->kvm = kvm;@@ -524,6 +663,7 @@ static unsigned long kvmppc_share_page(struct kvm *kvm, unsigned long gpa, uvmem_page =pfn_to_page (uvmem_pfn); pvt = uvmem_page->zone_device_data; pvt->skip_page_out = true; + pvt->remove_gfn = false; } retry:
arch/powerpc/kvm/book3s_hv_uvmem.c static unsigned long kvmppc_share_page(struct kvm *kvm, unsigned long gpa, · line 680
Quoted 4 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +1
-0@@ -537,12 +677,16 @@ static unsigned long kvmppc_share_page(struct kvm *kvm, unsigned long gpa, uvmem_page =pfn_to_page (uvmem_pfn); pvt = uvmem_page->zone_device_data; pvt->skip_page_out = true; + pvt->remove_gfn = false;
This is the case of making an already secure page as shared page. A comment here as to why remove_gfn is set to false here will help.
Also isn't it by default false? Is there a situation where it starts out by default false, becomes true later and you are required to explicitly mark it false here?
Otherwise, Reviewed-by: Bharata B Rao <[email protected]>
Regards,
Bharata.
Re: [v4 2/5] KVM: PPC: Book3S HV: track the state GFNs associated with secure VMs
Quoted 2 lines · Bharata B Rao
Quoted 1 line · Ram Pai
pvt->gpa= gpa;
..snip..
Quoted 18 lines
Quoted 10 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +1
-0pvt->kvm = kvm;@@ -524,6 +663,7 @@ static unsigned long kvmppc_share_page(struct kvm *kvm, unsigned long gpa, uvmem_page =pfn_to_page (uvmem_pfn); pvt = uvmem_page->zone_device_data; pvt->skip_page_out = true; + pvt->remove_gfn = false; } retry:arch/powerpc/kvm/book3s_hv_uvmem.c static unsigned long kvmppc_share_page(struct kvm *kvm, unsigned long gpa, · line 680
Quoted 4 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +1
-0@@ -537,12 +677,16 @@ static unsigned long kvmppc_share_page(struct kvm *kvm, unsigned long gpa, uvmem_page =pfn_to_page (uvmem_pfn); pvt = uvmem_page->zone_device_data; pvt->skip_page_out = true; + pvt->remove_gfn = false;This is the case of making an already secure page as shared page. A comment here as to why
remove_gfnis set to false here will help.
Quoted 3 lines
Also isn't it by default false? Is there a situation where it starts out by default false, becomes true later and you are required to explicitly mark it false here?
It is by default false. And will be true when the GFN is released/invalidated through kvmppc_uvmem_drop_pages().
It is marked false explicitly here, just to be safe, and protect against any implicit changes.
Quoted 3 lines
Otherwise, Reviewed-by: Bharata B Rao <[email protected]>
Thanks for the review.
RP
PATCH v4 3/5 KVM: PPC: Book3S HV: in H_SVM_INIT_DONE, migrate remaining normal-GFNs to secure-GFNs.
In reply to Ram PaiThe Ultravisor is expected to explicitly call H_SVM_PAGE_IN for all the pages of the SVM before calling H_SVM_INIT_DONE. This causes a huge delay in tranistioning the VM to SVM. The Ultravisor is only interested in the pages that contain the kernel, initrd and other important data structures. The rest contain throw-away content.
However if not all pages are requested by the Ultravisor, the Hypervisor continues to consider the GFNs corresponding to the non-requested pages as normal GFNs. This can lead to data-corruption and undefined behavior.
In H_SVM_INIT_DONE handler, move all the PFNs associated with the SVM's GFNs to secure-PFNs. Skip the GFNs that are already Paged-in or Shared or Paged-in followed by a Paged-out.
--- Documentation/powerpc/ultravisor.rst | 2 + arch/powerpc/include/asm/kvm_book3s_uvmem.h | 2 + arch/powerpc/kvm/book3s_hv_uvmem.c | 135 +++++++++++++++++++++++++--- 3 files changed, 125 insertions(+), 14 deletions(-)
Documentation/powerpc/ultravisor.rst +2 -0
@@ -934,6 +934,8 @@ Return values * H_UNSUPPORTED if called from the wrong context (e.g. from an SVM or before an H_SVM_INIT_START hypercall). + * H_STATE if the hypervisor could not successfully + transition the VM to Secure VM. Description ~~~~~~~~~~~
arch/powerpc/include/asm/kvm_book3s_uvmem.h +2 -0
@@ -23,6 +23,8 @@ unsigned long kvmppc_h_svm_page_out(struct kvm *kvm,unsigned long kvmppc_h_svm_init_abort (struct kvm *kvm);void kvmppc_uvmem_drop_pages (const struct kvm_memory_slot *free,struct kvm *kvm,bool skip_page_out); +int kvmppc_uv_migrate_mem_slot (struct kvm *kvm, +const struct kvm_memory_slot *memslot);#else static inline int kvmppc_uvmem_init (void ) {
arch/powerpc/kvm/book3s_hv_uvmem.c +121 -14
184 lines not on this page · raw message
Signature · 1 line
1.8.3.1
Re: [v4 3/5] KVM: PPC: Book3S HV: in H_SVM_INIT_DONE, migrate remaining normal-GFNs to secure-GFNs.
arch/powerpc/kvm/book3s_hv_uvmem.c static int kvmppc_svm_page_in(struct vm_area_struct *vma, unsigned long start, · line 704
Quoted 3 lines · Ram Pai
arch/powerpc/kvm/book3s_hv_uvmem.c +2
-0+int kvmppc_uv_migrate_mem_slot (struct kvm *kvm, +const struct kvm_memory_slot *memslot)
Don't see any callers for this outside of this file, so why not static?
arch/powerpc/kvm/book3s_hv_uvmem.c static int kvmppc_svm_page_in(struct vm_area_struct *vma, unsigned long start, · line 711
Quoted 7 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +7
-0+{ +unsigned long gfn = memslot->base_gfn; +struct vm_area_struct *vma; +unsigned long start, end; +int ret =0 ; + +while (kvmppc_next_nontransitioned_gfn (memslot, kvm, &gfn)) {
So you checked the state of gfn under uvmem_lock above, but release it too.
Quoted 8 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +8
-0+ +mmap_read_lock (kvm->mm); + start =gfn_to_hva (kvm, gfn); +if (kvm_is_error_hva (start)) { + ret =H_STATE ; + goto next; + } +
arch/powerpc/kvm/book3s_hv_uvmem.c static int kvmppc_svm_page_in(struct vm_area_struct *vma, unsigned long start, · line 729
Quoted 10 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +10
-0+ end = start + (1UL <<PAGE_SHIFT ); + vma =find_vma_intersection (kvm->mm, start, end); +if (!vma || vma->vm_start > start || vma->vm_end < end) { + ret =H_STATE ; + goto next; + } + +mutex_lock (&kvm->arch.uvmem_lock); + ret =kvmppc_svm_migrate_page (vma, start, end, + (gfn <<PAGE_SHIFT ), kvm,PAGE_SHIFT , false);
What is the guarantee that the gfn is in the same earlier state when you do do migration here?
Regards,
Bharata.
Re: [v4 3/5] KVM: PPC: Book3S HV: in H_SVM_INIT_DONE, migrate remaining normal-GFNs to secure-GFNs.
Quoted 31 lines · Bharata B Rao
arch/powerpc/kvm/book3s_hv_uvmem.c static int kvmppc_svm_page_in(struct vm_area_struct *vma, unsigned long start, · line 704
Quoted 3 lines · Ram Pai
arch/powerpc/kvm/book3s_hv_uvmem.c +2
-0+int kvmppc_uv_migrate_mem_slot (struct kvm *kvm, +const struct kvm_memory_slot *memslot)Don't see any callers for this outside of this file, so why not static?
arch/powerpc/kvm/book3s_hv_uvmem.c static int kvmppc_svm_page_in(struct vm_area_struct *vma, unsigned long start, · line 711
Quoted 7 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +7
-0+{ +unsigned long gfn = memslot->base_gfn; +struct vm_area_struct *vma; +unsigned long start, end; +int ret =0 ; + +while (kvmppc_next_nontransitioned_gfn (memslot, kvm, &gfn)) {So you checked the state of gfn under
uvmem_lockabove, but release it too.Quoted 13 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +13
-0+ +mmap_read_lock (kvm->mm); + start =gfn_to_hva (kvm, gfn); +if (kvm_is_error_hva (start)) { + ret =H_STATE ; + goto next; + } + + end = start + (1UL <<PAGE_SHIFT ); + vma =find_vma_intersection (kvm->mm, start, end); +if (!vma || vma->vm_start > start || vma->vm_end < end) { + ret =H_STATE ; + goto next;
Quoted 8 lines
arch/powerpc/kvm/book3s_hv_uvmem.c static int kvmppc_svm_page_in(struct vm_area_struct *vma, unsigned long start, · line 729
Quoted 5 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +5
-0+ } + +mutex_lock (&kvm->arch.uvmem_lock); + ret =kvmppc_svm_migrate_page (vma, start, end, + (gfn <<PAGE_SHIFT ), kvm,PAGE_SHIFT , false);What is the guarantee that the gfn is in the same earlier state when you do do migration here?
Are you worried about the case, where someother thread will sneak-in and migrate the GFN, and this migration request will become a duplicate one?
That is theortically possible, though practically improbable. This transition is attempted only when there is one vcpu active in the VM.
However, may be, we should not bake-in that assumption in this code. Will remove that assumption.
RP
PATCH v4 4/5 KVM: PPC: Book3S HV: retry page migration before erroring-out
In reply to Ram PaiThe page requested for page-in; sometimes, can have transient references, and hence cannot migrate immediately. Retry a few times before returning error.
The same is true for non-migrated pages that are migrated in H_SVM_INIT_DONE hanlder. Retry a few times before returning error.
H_SVM_PAGE_IN interface is enhanced to return H_BUSY if the page is not in a migratable state.
--- Documentation/powerpc/ultravisor.rst | 1 + arch/powerpc/kvm/book3s_hv_uvmem.c | 106 ++++++++++++++++++++++++----------- 2 files changed, 74 insertions(+), 33 deletions(-)
Documentation/powerpc/ultravisor.rst +1 -0
@@ -1035,6 +1035,7 @@ Return values * H_PARAMETER if ``guest_pa`` is invalid. * H_P2 if ``flags`` is invalid. * H_P3 if ``order`` of page is invalid. + * H_BUSY if ``page`` is not in a state to pagein Description ~~~~~~~~~~~
arch/powerpc/kvm/book3s_hv_uvmem.c +73 -33
164 lines not on this page · raw message
Signature · 1 line
1.8.3.1
Re: [v4 4/5] KVM: PPC: Book3S HV: retry page migration before erroring-out
Quoted 18 lines · Ram Pai
arch/powerpc/kvm/book3s_hv_uvmem.c +1
-4@@ -812,7 +842,7 @@ unsigned long kvmppc_h_svm_page_in(struct kvm *kvm, unsigned long gpa,struct vm_area_struct *vma;int srcu_idx;unsigned long gfn = gpa >> page_shift; -int ret; +int ret, repeat_count =REPEAT_COUNT ;if (!(kvm->arch.secure_guest &KVMPPC_SECURE_INIT_START ))return H_UNSUPPORTED ;@@ -826,34 +856,44 @@ unsigned long kvmppc_h_svm_page_in(struct kvm *kvm, unsigned long gpa,if (flags &H_PAGE_IN_SHARED )return kvmppc_share_page (kvm, gpa, page_shift); - ret =H_PARAMETER ; srcu_idx =srcu_read_lock (&kvm->srcu); -mmap_read_lock (kvm->mm); - start =gfn_to_hva (kvm, gfn);
arch/powerpc/kvm/book3s_hv_uvmem.c unsigned long kvmppc_h_svm_page_in(struct kvm *kvm, unsigned long gpa, · line 864
Quoted 10 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +3
-6-if (kvm_is_error_hva (start)) - goto out; - -mutex_lock (&kvm->arch.uvmem_lock);/* Fail the page-in request of an already paged-in page */ -if (kvmppc_gfn_is_uvmem_pfn (gfn, kvm,NULL )) - goto out_unlock; +mutex_lock (&kvm->arch.uvmem_lock); + ret =kvmppc_gfn_is_uvmem_pfn (gfn, kvm,NULL ); +mutex_unlock (&kvm->arch.uvmem_lock);
Same comment as for the prev patch. I don't think you can release the lock here.
Quoted 49 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +28
-12+if (ret) { +srcu_read_unlock (&kvm->srcu, srcu_idx); +return H_PARAMETER ; + } - end = start + (1UL << page_shift); - vma =find_vma_intersection (kvm->mm, start, end); -if (!vma || vma->vm_start > start || vma->vm_end < end) - goto out_unlock; +do { + ret =H_PARAMETER ; +mmap_read_lock (kvm->mm); -if (kvmppc_svm_migrate_page (vma, start, end, gpa, kvm, page_shift, - true)) - goto out_unlock; + start =gfn_to_hva (kvm, gfn); +if (kvm_is_error_hva (start)) { +mmap_read_unlock (kvm->mm); +break ; + } - ret =H_SUCCESS ; + end = start + (1UL << page_shift); + vma =find_vma_intersection (kvm->mm, start, end); +if (!vma || vma->vm_start > start || vma->vm_end < end) { +mmap_read_unlock (kvm->mm); +break ; + } + +mutex_lock (&kvm->arch.uvmem_lock); + ret =kvmppc_svm_migrate_page (vma, start, end, gpa, kvm, page_shift, true); +mutex_unlock (&kvm->arch.uvmem_lock); + +mmap_read_unlock (kvm->mm); + }while (ret ==-2 && repeat_count--); + +if (ret ==-2 ) + ret =H_BUSY ; -out_unlock: -mutex_unlock (&kvm->arch.uvmem_lock); -out: -mmap_read_unlock (kvm->mm);srcu_read_unlock (&kvm->srcu, srcu_idx);return ret; }Signature · 1 line
1.8.3.1
Re: [v4 4/5] KVM: PPC: Book3S HV: retry page migration before erroring-out
I am dropping this patch based on our conversation, where we agreed, we need to rootcause the migration failure.
Quoted 82 lines · Bharata B Rao
Quoted 18 lines · Ram Pai
arch/powerpc/kvm/book3s_hv_uvmem.c +1
-4@@ -812,7 +842,7 @@ unsigned long kvmppc_h_svm_page_in(struct kvm *kvm, unsigned long gpa,struct vm_area_struct *vma;int srcu_idx;unsigned long gfn = gpa >> page_shift; -int ret; +int ret, repeat_count =REPEAT_COUNT ;if (!(kvm->arch.secure_guest &KVMPPC_SECURE_INIT_START ))return H_UNSUPPORTED ;@@ -826,34 +856,44 @@ unsigned long kvmppc_h_svm_page_in(struct kvm *kvm, unsigned long gpa,if (flags &H_PAGE_IN_SHARED )return kvmppc_share_page (kvm, gpa, page_shift); - ret =H_PARAMETER ; srcu_idx =srcu_read_lock (&kvm->srcu); -mmap_read_lock (kvm->mm); - start =gfn_to_hva (kvm, gfn);arch/powerpc/kvm/book3s_hv_uvmem.c unsigned long kvmppc_h_svm_page_in(struct kvm *kvm, unsigned long gpa, · line 864
Quoted 10 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +3
-6-if (kvm_is_error_hva (start)) - goto out; - -mutex_lock (&kvm->arch.uvmem_lock);/* Fail the page-in request of an already paged-in page */ -if (kvmppc_gfn_is_uvmem_pfn (gfn, kvm,NULL )) - goto out_unlock; +mutex_lock (&kvm->arch.uvmem_lock); + ret =kvmppc_gfn_is_uvmem_pfn (gfn, kvm,NULL ); +mutex_unlock (&kvm->arch.uvmem_lock);Same comment as for the prev patch. I don't think you can release the lock here.
Quoted 49 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +28
-12+if (ret) { +srcu_read_unlock (&kvm->srcu, srcu_idx); +return H_PARAMETER ; + } - end = start + (1UL << page_shift); - vma =find_vma_intersection (kvm->mm, start, end); -if (!vma || vma->vm_start > start || vma->vm_end < end) - goto out_unlock; +do { + ret =H_PARAMETER ; +mmap_read_lock (kvm->mm); -if (kvmppc_svm_migrate_page (vma, start, end, gpa, kvm, page_shift, - true)) - goto out_unlock; + start =gfn_to_hva (kvm, gfn); +if (kvm_is_error_hva (start)) { +mmap_read_unlock (kvm->mm); +break ; + } - ret =H_SUCCESS ; + end = start + (1UL << page_shift); + vma =find_vma_intersection (kvm->mm, start, end); +if (!vma || vma->vm_start > start || vma->vm_end < end) { +mmap_read_unlock (kvm->mm); +break ; + } + +mutex_lock (&kvm->arch.uvmem_lock); + ret =kvmppc_svm_migrate_page (vma, start, end, gpa, kvm, page_shift, true); +mutex_unlock (&kvm->arch.uvmem_lock); + +mmap_read_unlock (kvm->mm); + }while (ret =-2 && repeat_count--); + +if (ret =-2 ) + ret =H_BUSY ; -out_unlock: -mutex_unlock (&kvm->arch.uvmem_lock); -out: -mmap_read_unlock (kvm->mm);srcu_read_unlock (&kvm->srcu, srcu_idx);return ret; }Signature · 1 line
1.8.3.1
Signature · 1 line
Ram Pai
PATCH v4 5/5 KVM: PPC: Book3S HV: migrate hot plugged memory
In reply to Ram PaiFrom: Laurent Dufour <[email protected]>
When a memory slot is hot plugged to a SVM, PFNs associated with the GFNs in that slot must be migrated to secure-PFNs, aka device-PFNs.
Call kvmppc_uv_migrate_mem_slot() to accomplish this. Disable page-merge for all pages in the memory slot.
--- arch/powerpc/include/asm/kvm_book3s_uvmem.h | 10 ++++++++++ arch/powerpc/kvm/book3s_hv.c | 10 ++-------- arch/powerpc/kvm/book3s_hv_uvmem.c | 22 ++++++++++++++++++++++ 3 files changed, 34 insertions(+), 8 deletions(-)
arch/powerpc/include/asm/kvm_book3s_uvmem.h +10 -0
@@ -25,6 +25,9 @@ void kvmppc_uvmem_drop_pages(const struct kvm_memory_slot *free, struct kvm *kvm, bool skip_page_out);int kvmppc_uv_migrate_mem_slot (struct kvm *kvm,const struct kvm_memory_slot *memslot); +void kvmppc_memslot_create (struct kvm *kvm,const struct kvm_memory_slot *new); +void kvmppc_memslot_delete (struct kvm *kvm,const struct kvm_memory_slot *old); +#else static inline int kvmppc_uvmem_init (void ) {
@@ -84,5 +87,12 @@ static inline int kvmppc_send_page_to_uv(struct kvm *kvm, unsigned long gfn)static inline void kvmppc_uvmem_drop_pages (const struct kvm_memory_slot *free,struct kvm *kvm,bool skip_page_out) { } + +static inline void kvmppc_memslot_create (struct kvm *kvm, +const struct kvm_memory_slot *new) { } + +static inline void kvmppc_memslot_delete (struct kvm *kvm, +const struct kvm_memory_slot *old) { } +#endif /* CONFIG_PPC_UV */ #endif /* __ASM_KVM_BOOK3S_UVMEM_H__ */
arch/powerpc/kvm/book3s_hv.c +2 -8
@@ -4515,16 +4515,10 @@ static void kvmppc_core_commit_memory_region_hv(struct kvm *kvm,switch (change) {case KVM_MR_CREATE : -if (kvmppc_uvmem_slot_init (kvm, new)) -return ; -uv_register_mem_slot (kvm->arch.lpid, - new->base_gfn <<PAGE_SHIFT , - new->npages *PAGE_SIZE , -0 , new->id); +kvmppc_memslot_create (kvm, new);break ;case KVM_MR_DELETE : -uv_unregister_mem_slot (kvm->arch.lpid, old->id); -kvmppc_uvmem_slot_free (kvm, old); +kvmppc_memslot_delete (kvm, old);break ;default :/* TODO: Handle KVM_MR_MOVE */
arch/powerpc/kvm/book3s_hv_uvmem.c +22 -0
@@ -1089,6 +1089,28 @@ int kvmppc_send_page_to_uv(struct kvm *kvm, unsigned long gfn)return (ret =U_SUCCESS ) ?RESUME_GUEST : -EFAULT ; } +void kvmppc_memslot_create (struct kvm *kvm,const struct kvm_memory_slot *new) +{ +if (kvmppc_uvmem_slot_init (kvm, new)) +return ; + +if (kvmppc_memslot_page_merge (kvm, new, false)) +return ; + +if (uv_register_mem_slot (kvm->arch.lpid, new->base_gfn <<PAGE_SHIFT , + new->npages *PAGE_SIZE ,0 , new->id)) +return ; + +kvmppc_uv_migrate_mem_slot (kvm, new); +} + +void kvmppc_memslot_delete (struct kvm *kvm,const struct kvm_memory_slot *old) +{ +uv_unregister_mem_slot (kvm->arch.lpid, old->id); +kvmppc_memslot_page_merge (kvm, old, true); +kvmppc_uvmem_slot_free (kvm, old); +} +static u64 kvmppc_get_secmem_size (void ) {struct device_node *np;
Signature · 1 line
1.8.3.1
Re: [v4 5/5] KVM: PPC: Book3S HV: migrate hot plugged memory
Quoted 22 lines · Ram Pai
From: Laurent Dufour <[email protected]>
When a memory slot is hot plugged to a SVM, PFNs associated with the GFNs in that slot must be migrated to secure-PFNs, aka device-PFNs.
Call
kvmppc_uv_migrate_mem_slot()to accomplish this. Disable page-merge for all pages in the memory slot.Signed-off-by Ram Pai <[email protected]>[rearranged the code, and modified the commit log]Signed-off-by Laurent Dufour <[email protected]>--- arch/powerpc/include/asm/kvm_book3s_uvmem.h | 10 ++++++++++ arch/powerpc/kvm/book3s_hv.c | 10 ++-------- arch/powerpc/kvm/book3s_hv_uvmem.c | 22 ++++++++++++++++++++++ 3 files changed, 34 insertions(+), 8 deletions(-)arch/powerpc/include/asm/kvm_book3s_uvmem.h +0
-0
arch/powerpc/include/asm/kvm_book3s_uvmem.h void kvmppc_uvmem_drop_pages(const struct kvm_memory_slot *free, · line 29
Quoted 5 lines
arch/powerpc/include/asm/kvm_book3s_uvmem.h +2
-0@@ -25,6 +25,9 @@ void kvmppc_uvmem_drop_pages(const struct kvm_memory_slot *free, struct kvm *kvm, bool skip_page_out);int kvmppc_uv_migrate_mem_slot (struct kvm *kvm,const struct kvm_memory_slot *memslot); +void kvmppc_memslot_create (struct kvm *kvm,const struct kvm_memory_slot *new); +void kvmppc_memslot_delete (struct kvm *kvm,const struct kvm_memory_slot *old);
The names look a bit generic, but these functions are specific
to secure guests. May be rename them to kvmppc_uvmem_memslot_[create/delele]?
Quoted 51 lines
arch/powerpc/include/asm/kvm_book3s_uvmem.h +8
-0+#else static inline int kvmppc_uvmem_init (void ) {@@ -84,5 +87,12 @@ static inline int kvmppc_send_page_to_uv(struct kvm *kvm, unsigned long gfn)static inline void kvmppc_uvmem_drop_pages (const struct kvm_memory_slot *free,struct kvm *kvm,bool skip_page_out) { } + +static inline void kvmppc_memslot_create (struct kvm *kvm, +const struct kvm_memory_slot *new) { } + +static inline void kvmppc_memslot_delete (struct kvm *kvm, +const struct kvm_memory_slot *old) { } +#endif /* CONFIG_PPC_UV */ #endif /* __ASM_KVM_BOOK3S_UVMEM_H__ */ arch/powerpc/kvm/book3s_hv.c +2
-8@@ -4515,16 +4515,10 @@ static void kvmppc_core_commit_memory_region_hv(struct kvm *kvm,switch (change) {case KVM_MR_CREATE : -if (kvmppc_uvmem_slot_init (kvm, new)) -return ; -uv_register_mem_slot (kvm->arch.lpid, - new->base_gfn <<PAGE_SHIFT , - new->npages *PAGE_SIZE , -0 , new->id); +kvmppc_memslot_create (kvm, new);break ;case KVM_MR_DELETE : -uv_unregister_mem_slot (kvm->arch.lpid, old->id); -kvmppc_uvmem_slot_free (kvm, old); +kvmppc_memslot_delete (kvm, old);break ;default :/* TODO: Handle KVM_MR_MOVE */ arch/powerpc/kvm/book3s_hv_uvmem.c +3
-0@@ -1089,6 +1089,28 @@ int kvmppc_send_page_to_uv(struct kvm *kvm, unsigned long gfn)return (ret ==U_SUCCESS ) ?RESUME_GUEST : -EFAULT ; } +void kvmppc_memslot_create (struct kvm *kvm,const struct kvm_memory_slot *new) +{ +if (kvmppc_uvmem_slot_init (kvm, new))
arch/powerpc/kvm/book3s_hv_uvmem.c int kvmppc_send_page_to_uv(struct kvm *kvm, unsigned long gfn) · line 1104
Quoted 10 lines
arch/powerpc/kvm/book3s_hv_uvmem.c +10
-0+return ; + +if (kvmppc_memslot_page_merge (kvm, new, false)) +return ; + +if (uv_register_mem_slot (kvm->arch.lpid, new->base_gfn <<PAGE_SHIFT , + new->npages *PAGE_SIZE ,0 , new->id)) +return ; + +kvmppc_uv_migrate_mem_slot (kvm, new);
Quite a few things can return failure here including kvmppc_uv_migrate_mem_slot() and we are ignoring all of those. I am wondering if this should be called from prepare_memory_region callback instead of commit_memory_region. In the prepare phase, we have a way to back out in case of error. Can you check if moving this call to prepare callback is feasible?
In the other case in 1/5, the code issues ksm unmerge request on error, but not here.
Also check if the code for 1st three calls can be shared with similar code in 1/5.
Regards,
Bharata.