KVM: s390: pv: implement lazy destroy
Claudio Imbrenda
maintainerv2 cover KVM: s390: pv: implement lazy destroy
Previously, when a protected VM was rebooted or when it was shut down, its memory was made unprotected, and then the protected VM itself was destroyed. Looping over the whole address space can take some time, considering the overhead of the various Ultravisor Calls (UVCs). This means that a reboot or a shutdown would take a potentially long amount of time, depending on the amount of used memory.
This patchseries implements a deferred destroy mechanism for protected guests. When a protected guest is destroyed, its memory is cleared in background, allowing the guest to restart or terminate significantly faster than before.
There are 2 possibilities when a protected VM is torn down:
* it still has an address space associated (reboot case)
* it does not have an address space anymore (shutdown case)
For the reboot case, the reference count of the mm is increased, and then a background thread is started to clean up. Once the thread went through the whole address space, the protected VM is actually destroyed.
For the shutdown case, a list of pages to be destroyed is formed when the mm is torn down. Instead of just unmapping the pages when the address space is being torn down, they are also set aside. Later when KVM cleans up the VM, a thread is started to clean up the pages from the list.
This means that the same address space can have memory belonging to more than one protected guest, although only one will be running, the others will in fact not even have any CPUs.
When a guest is destroyed, its memory still counts towards its memory control group until it's actually freed (I tested this experimentally)
When the system runs out of memory, if a guest has terminated and its memory is being cleaned asynchronously, the OOM killer will wait a little and then see if memory has been freed. This has the practical effect of slowing down memory allocations when the system is out of memory to give the cleanup thread time to cleanup and free memory, and avoid an actual OOM situation.
v1->v2
* rebased on a more recent kernel
* improved/expanded some patch descriptions
* improves/expanded some comments
* added patch 1, which prevents stall notification when the system is under heavy load.
* rename some members of struct deferred_priv to improve readability
* avoid an use-after-free bug of the struct mm in case of shutdown
* add missing return when lazy destroy is disabled
* add support for OOM notifier
Claudio Imbrenda (13):
KVM: s390: pv: avoid stall notifications for some UVCs
KVM: s390: pv: leak the ASCE page when destroy fails
KVM: s390: pv: properly handle page flags for protected guests
KVM: s390: pv: handle secure storage violations for protected guests
KVM: s390: pv: handle secure storage exceptions for normal guests
KVM: s390: pv: refactor s390_reset_acc
KVM: s390: pv: usage counter instead of flag
KVM: s390: pv: add export before import
KVM: s390: pv: lazy destroy for reboot
KVM: s390: pv: extend lazy destroy to handle shutdown
KVM: s390: pv: module parameter to fence lazy destroy
KVM: s390: pv: add OOM notifier for lazy destroy
KVM: s390: pv: add support for UV feature bits
arch/s390/include/asm/gmap.h | 5 +- arch/s390/include/asm/mmu.h | 3 + arch/s390/include/asm/mmu_context.h | 2 + arch/s390/include/asm/pgtable.h | 16 +- arch/s390/include/asm/uv.h | 26 ++- arch/s390/kernel/uv.c | 144 ++++++++++++++- arch/s390/kvm/kvm-s390.c | 6 +- arch/s390/kvm/kvm-s390.h | 2 +- arch/s390/kvm/pv.c | 272 ++++++++++++++++++++++++++-- arch/s390/mm/fault.c | 22 ++- arch/s390/mm/gmap.c | 86 ++++++--- 11 files changed, 530 insertions(+), 54 deletions(-)
Signature · 1 line
2.31.1
Claudio Imbrenda
maintainerPATCH v2 1/13 KVM: s390: pv: avoid stall notifications for some UVCs
Improve make_secure_pte to avoid stalls when the system is heavily overcommitted. This was especially problematic in kvm_s390_pv_unpack, because of the loop over all pages that needed unpacking.
Also fix kvm_s390_pv_init_vm to avoid stalls when the system is heavily overcommitted.
--- arch/s390/kernel/uv.c | 11 ++++++++--- arch/s390/kvm/pv.c | 2 +- 2 files changed, 9 insertions(+), 4 deletions(-)
arch/s390/kernel/uv.c +8 -3
@@ -196,11 +196,16 @@ static int make_secure_pte(pte_t *ptep, unsigned long addr,if (!page_ref_freeze (page, expected))return -EBUSY ;set_bit (PG_arch_1, &page->flags); - rc =uv_call (0 , (u64 )uvcb); + rc =__uv_call (0 , (u64 )uvcb);page_ref_unfreeze (page, expected); -/* Return -ENXIO if the page was not mapped, -EINVAL otherwise */ -if (rc) +/* +* Return -ENXIO if the page was not mapped, -EINVAL for other errors. +* If busy or partially completed, return -EAGAIN. +*/ +if (rc ==1 ) rc = uvcb->rc ==0x10a ? -ENXIO : -EINVAL ; +else if (rc >1 ) + rc = -EAGAIN ;return rc; }
arch/s390/kvm/pv.c +1 -1
@@ -196,7 +196,7 @@ int kvm_s390_pv_init_vm(struct kvm *kvm, u16 *rc, u16 *rrc) uvcb.conf_base_stor_origin = (u64 )kvm->arch.pv.stor_base; uvcb.conf_virt_stor_origin = (u64 )kvm->arch.pv.stor_var; - cc =uv_call (0 , (u64)&uvcb); + cc =uv_call_sched (0 , (u64)&uvcb); *rc = uvcb.header.rc; *rrc = uvcb.header.rrc; KVM_UV_EVENT(kvm, 3, "PROTVIRT CREATE VM: handle %llx len %llx rc %x rrc %x",
Signature · 1 line
2.31.1
Janosch Frank
maintainerRe: [PATCH v2 01/13] KVM: s390: pv: avoid stall notifications for some UVCs
Quoted 6 lines · Claudio Imbrenda
Improve
make_secure_pteto avoid stalls when the system is heavily overcommitted. This was especially problematic inkvm_s390_pv_unpack, because of the loop over all pages that needed unpacking.Also fix
kvm_s390_pv_init_vmto avoid stalls when the system is heavily overcommitted.
Fixes tag?
Quoted 12 lines
Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/kernel/uv.c | 11 ++++++++--- arch/s390/kvm/pv.c | 2 +- 2 files changed, 9 insertions(+), 4 deletions(-)arch/s390/kernel/uv.c +0
-0
arch/s390/kernel/uv.c static int make_secure_pte(pte_t *ptep, unsigned long addr, · line 199
Quoted 5 lines
arch/s390/kernel/uv.c +1
-1@@ -196,11 +196,16 @@ static int make_secure_pte(pte_t *ptep, unsigned long addr,if (!page_ref_freeze (page, expected))return -EBUSY ;set_bit (PG_arch_1, &page->flags); - rc =uv_call (0 , (u64 )uvcb); + rc =__uv_call (0 , (u64 )uvcb);
We should exchange rc with cc since that's what we get back from __uv_call(). Technically we always get a cc but for the other functions it's only ever 0/1 which translates to success/error so rc is ok.
arch/s390/kernel/uv.c static int make_secure_pte(pte_t *ptep, unsigned long addr, · line 209
Quoted 12 lines
arch/s390/kernel/uv.c +7
-2page_ref_unfreeze (page, expected); -/* Return -ENXIO if the page was not mapped, -EINVAL otherwise */ -if (rc) +/* +* Return -ENXIO if the page was not mapped, -EINVAL for other errors. +* If busy or partially completed, return -EAGAIN. +*/ +if (rc ==1 ) rc = uvcb->rc ==0x10a ? -ENXIO : -EINVAL ; +else if (rc >1 ) + rc = -EAGAIN ;return rc;
Could you define the CCs in uv.h and check against the constants here so it's easier to understand that the rc > 1 checks against a "UV was busy please re-issue the call again" cc?
Maybe also make it explicit for cc 2 and 3 instead of cc > 1
Quoted 16 lines
arch/s390/kernel/uv.c +0
-0}arch/s390/kvm/pv.c +1
-1@@ -196,7 +196,7 @@ int kvm_s390_pv_init_vm(struct kvm *kvm, u16 *rc, u16 *rrc) uvcb.conf_base_stor_origin = (u64 )kvm->arch.pv.stor_base; uvcb.conf_virt_stor_origin = (u64 )kvm->arch.pv.stor_var; - cc =uv_call (0 , (u64)&uvcb); + cc =uv_call_sched (0 , (u64)&uvcb); *rc = uvcb.header.rc; *rrc = uvcb.header.rrc; KVM_UV_EVENT(kvm, 3, "PROTVIRT CREATE VM: handle %llx len %llx rc %x rrc %x",
Claudio Imbrenda
maintainerRe: [PATCH v2 01/13] KVM: s390: pv: avoid stall notifications for some UVCs
Quoted 10 lines · On Thu, 29 Jul 2021 11:58:39 +0200 Janosch Frank <[email protected]> wrote:
Quoted 7 lines · Claudio Imbrenda
Improve
make_secure_pteto avoid stalls when the system is heavily overcommitted. This was especially problematic inkvm_s390_pv_unpack, because of the loop over all pages that needed unpacking.Also fix
kvm_s390_pv_init_vmto avoid stalls when the system is heavily overcommitted.Fixes tag?
will be in the next version
Quoted 7 lines
Quoted 7 lines
Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/kernel/uv.c | 11 ++++++++--- arch/s390/kvm/pv.c | 2 +- 2 files changed, 9 insertions(+), 4 deletions(-)
Quoted 15 lines
arch/s390/kernel/uv.c static int make_secure_pte(pte_t *ptep, · line 199
Quoted 10 lines
arch/s390/kernel/uv.c +1
-1@@ -196,11 +196,16 @@ static int make_secure_pte(pte_t *ptep, unsigned long addr, if (!page_ref_freeze(page, expected)) return -EBUSY;set_bit (PG_arch_1, &page->flags); - rc =uv_call (0 , (u64 )uvcb); + rc =__uv_call (0 , (u64 )uvcb);We should exchange rc with cc since that's what we get back from
__uv_call(). Technically we always get a cc but for the other functions it's only ever 0/1 which translates to success/error so rc is ok.
will be in the next version
Quoted 19 lines
Quoted 7 lines
arch/s390/kernel/uv.c +0
-1page_ref_unfreeze (page, expected); -/* Return -ENXIO if the page was not mapped, -EINVAL otherwise */
arch/s390/kernel/uv.c +2
-1-if (rc) +/* +* Return -ENXIO if the page was not mapped, -EINVAL for other errors.
arch/s390/kernel/uv.c static int make_secure_pte(pte_t *ptep, unsigned long addr, · line 209
Quoted 7 lines
arch/s390/kernel/uv.c +5
-0+* If busy or partially completed, return -EAGAIN. +*/ +if (rc ==1 ) rc = uvcb->rc ==0x10a ? -ENXIO : -EINVAL ; +else if (rc >1 ) + rc = -EAGAIN ;return rc;Could you define the CCs in
uv.hand check against the constants here so it's easier to understand that the rc > 1 checks against a "UV was busy please re-issue the call again" cc?
Quoted 1 line
Maybe also make it explicit for cc 2 and 3 instead of cc > 1
will be in the next version
Quoted 17 lines
Quoted 16 lines
}
arch/s390/kvm/pv.c +0
-0@@ -196,7 +196,7 @@ int kvm_s390_pv_init_vm(struct kvm *kvm, u16 *rc, u16 *rrc) uvcb.conf_base_stor_origin =(u64)kvm->arch.pv.stor_base; uvcb.conf_virt_stor_origin = (u64)kvm->arch.pv.stor_var;
arch/s390/kvm/pv.c +1
-1- cc =uv_call (0 , (u64)&uvcb); + cc =uv_call_sched (0 , (u64)&uvcb); *rc = uvcb.header.rc; *rrc = uvcb.header.rrc; KVM_UV_EVENT(kvm, 3, "PROTVIRT CREATE VM: handle %llx len%llx rc %x rrc %x",
Re: [PATCH v2 01/13] KVM: s390: pv: avoid stall notifications for some UVCs
In reply to Claudio ImbrendaQuoted 26 lines · Claudio Imbrenda
Improve
make_secure_pteto avoid stalls when the system is heavily overcommitted. This was especially problematic inkvm_s390_pv_unpack, because of the loop over all pages that needed unpacking.Also fix
kvm_s390_pv_init_vmto avoid stalls when the system is heavily overcommitted.Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/kernel/uv.c | 11 ++++++++--- arch/s390/kvm/pv.c | 2 +- 2 files changed, 9 insertions(+), 4 deletions(-)arch/s390/kernel/uv.c +1
-3@@ -196,11 +196,16 @@ static int make_secure_pte(pte_t *ptep, unsigned long addr,if (!page_ref_freeze (page, expected))return -EBUSY ;set_bit (PG_arch_1, &page->flags); - rc =uv_call (0 , (u64 )uvcb); + rc =__uv_call (0 , (u64 )uvcb);page_ref_unfreeze (page, expected); -/* Return -ENXIO if the page was not mapped, -EINVAL otherwise */ -if (rc)
arch/s390/kernel/uv.c static int make_secure_pte(pte_t *ptep, unsigned long addr, · line 210
Quoted 10 lines
arch/s390/kernel/uv.c +7
-0+/* +* Return -ENXIO if the page was not mapped, -EINVAL for other errors. +* If busy or partially completed, return -EAGAIN. +*/ +if (rc ==1 ) rc = uvcb->rc ==0x10a ? -ENXIO : -EINVAL ; +else if (rc >1 ) + rc = -EAGAIN ;return rc; }
Possibly dumb question: when does the call return > 1? gmap_make_secure() will do a wait_on_page_writeback() for -EAGAIN, is that always the right thing to do?
Claudio Imbrenda
maintainerRe: [PATCH v2 01/13] KVM: s390: pv: avoid stall notifications for some UVCs
Quoted 35 lines · On Thu, 29 Jul 2021 12:49:03 +0200 Cornelia Huck <[email protected]> wrote:
Quoted 33 lines · Claudio Imbrenda
Improve
make_secure_pteto avoid stalls when the system is heavily overcommitted. This was especially problematic inkvm_s390_pv_unpack, because of the loop over all pages that needed unpacking.Also fix
kvm_s390_pv_init_vmto avoid stalls when the system is heavily overcommitted.Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/kernel/uv.c | 11 ++++++++--- arch/s390/kvm/pv.c | 2 +- 2 files changed, 9 insertions(+), 4 deletions(-)arch/s390/kernel/uv.c +1
-2@@ -196,11 +196,16 @@ static int make_secure_pte(pte_t *ptep, unsigned long addr, if (!page_ref_freeze(page, expected)) return -EBUSY;set_bit (PG_arch_1, &page->flags); - rc =uv_call (0 , (u64 )uvcb); + rc =__uv_call (0 , (u64 )uvcb);page_ref_unfreeze (page, expected); -/* Return -ENXIO if the page was not mapped, -EINVAL otherwise */
arch/s390/kernel/uv.c +2
-1-if (rc) +/* +* Return -ENXIO if the page was not mapped, -EINVAL for other errors.
arch/s390/kernel/uv.c +2
-0+* If busy or partially completed, return -EAGAIN. +*/
Quoted 8 lines
arch/s390/kernel/uv.c static int make_secure_pte(pte_t *ptep, unsigned long addr, · line 210
Quoted 6 lines
arch/s390/kernel/uv.c +3
-0+if (rc ==1 ) rc = uvcb->rc ==0x10a ? -ENXIO : -EINVAL ; +else if (rc >1 ) + rc = -EAGAIN ;return rc; }Possibly dumb question: when does the call return > 1?
this is exactly what Janosch meant :)
the next version will have #defines for the 4 possible CC values.
in short:
0 OK
1 error
2 busy (nothing done, try again)
3 partial (something done but not all, try again)
Quoted 2 lines
gmap_make_secure()will do await_on_page_writeback()for-EAGAIN, is that always the right thing to do?
it's the easiest way to get to a place where we will be able to reschedule if needed.
wait_on_page_writeback will probably do nothing in that case because the page is not in writeback.
(a few minutes later)
actually I have checked, it seems that the -EAGAIN gets eventually propagated to places where it's not checked properly!
this will need some more fixing
Claudio Imbrenda
maintainerPATCH v2 2/13 KVM: s390: pv: leak the ASCE page when destroy fails
In reply to Claudio ImbrendaWhen a protected VM is created, the topmost level of page tables of its ASCE is marked by the Ultravisor; any attempt to use that memory for protected virtualization will result in failure.
Only a successful Destroy Configuration UVC will remove the marking.
When the Destroy Configuration UVC fails, the topmost level of page tables of the VM does not get its marking cleared; to avoid issues it must not be used again.
Since the page becomes in practice unusable, we set it aside and leak it.
--- arch/s390/kvm/pv.c | 53 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 52 insertions(+), 1 deletion(-)
arch/s390/kvm/pv.c +52 -1
@@ -155,6 +155,55 @@ static int kvm_s390_pv_alloc_vm(struct kvm *kvm)return -ENOMEM ; } +/* +* Remove the topmost level of page tables from the list of page tables of +* the gmap. +* This means that it will not be freed when the VM is torn down, and needs +* to be handled separately by the caller, unless an intentional leak is +* intended. +*/ +static void kvm_s390_pv_remove_old_asce (struct kvm *kvm) +{ +struct page *old; + + old =virt_to_page (kvm->arch.gmap->table); +list_del (&old->lru); +/* in case the ASCE needs to be "removed" multiple times */ +INIT_LIST_HEAD (&old->lru); +} + +/* +* Try to replace the current ASCE with another equivalent one. +* If the allocation of the new top level page table fails, the ASCE is not +* replaced. +* In any case, the old ASCE is removed from the list, therefore the caller +* has to make sure to save a pointer to it beforehands, unless an +* intentional leak is intended. +*/ +static int kvm_s390_pv_replace_asce (struct kvm *kvm) +{ +unsigned long asce; +struct page *page; +void *table; + +kvm_s390_pv_remove_old_asce (kvm); + + page =alloc_pages (GFP_KERNEL_ACCOUNT ,CRST_ALLOC_ORDER ); +if (!page) +return -ENOMEM ; +list_add (&page->lru, &kvm->arch.gmap->crst_list); + + table =page_to_virt (page); +memcpy (table, kvm->arch.gmap->table,1UL << (CRST_ALLOC_ORDER +PAGE_SHIFT )); + + asce = (kvm->arch.gmap->asce & ~PAGE_MASK ) |__pa (table); +WRITE_ONCE (kvm->arch.gmap->asce, asce); +WRITE_ONCE (kvm->mm->context.gmap_asce, asce); +WRITE_ONCE (kvm->arch.gmap->table, table); + +return 0 ; +} +/* this should not fail, but if it does, we must not free the donated memory */ int kvm_s390_pv_deinit_vm (struct kvm *kvm,u16 *rc,u16 *rrc) {
@@ -169,9 +218,11 @@ int kvm_s390_pv_deinit_vm(struct kvm *kvm, u16 *rc, u16 *rrc)atomic_set (&kvm->mm->context.is_protected,0 );KVM_UV_EVENT (kvm,3 ,"PROTVIRT DESTROY VM: rc %x rrc %x" , *rc, *rrc);WARN_ONCE (cc,"protvirt destroy vm failed rc %x rrc %x" , *rc, *rrc); -/* Inteded memory leak on "impossible" error */ +/* Intended memory leak on "impossible" error */ if (!cc)kvm_s390_pv_dealloc_vm (kvm); +else +kvm_s390_pv_replace_asce (kvm);return cc ? -EIO :0 ; }
Signature · 1 line
2.31.1
Janosch Frank
maintainerRe: [PATCH v2 02/13] KVM: s390: pv: leak the ASCE page when destroy fails
Quoted 55 lines · Claudio Imbrenda
When a protected VM is created, the topmost level of page tables of its ASCE is marked by the Ultravisor; any attempt to use that memory for protected virtualization will result in failure.
Only a successful Destroy Configuration UVC will remove the marking.
When the Destroy Configuration UVC fails, the topmost level of page tables of the VM does not get its marking cleared; to avoid issues it must not be used again.
Since the page becomes in practice unusable, we set it aside and leak it.
Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/kvm/pv.c | 53 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 52 insertions(+), 1 deletion(-)arch/s390/kvm/pv.c +30
-0@@ -155,6 +155,55 @@ static int kvm_s390_pv_alloc_vm(struct kvm *kvm)return -ENOMEM ; } +/* +* Remove the topmost level of page tables from the list of page tables of +* the gmap. +* This means that it will not be freed when the VM is torn down, and needs +* to be handled separately by the caller, unless an intentional leak is +* intended. +*/ +static void kvm_s390_pv_remove_old_asce (struct kvm *kvm) +{ +struct page *old; + + old =virt_to_page (kvm->arch.gmap->table); +list_del (&old->lru); +/* in case the ASCE needs to be "removed" multiple times */ +INIT_LIST_HEAD (&old->lru); +} + +/* +* Try to replace the current ASCE with another equivalent one. +* If the allocation of the new top level page table fails, the ASCE is not +* replaced. +* In any case, the old ASCE is removed from the list, therefore the caller +* has to make sure to save a pointer to it beforehands, unless an +* intentional leak is intended. +*/ +static int kvm_s390_pv_replace_asce (struct kvm *kvm) +{ +unsigned long asce; +struct page *page; +void *table;
arch/s390/kvm/pv.c static int kvm_s390_pv_alloc_vm(struct kvm *kvm) · line 197
Quoted 10 lines
arch/s390/kvm/pv.c +10
-0+ +kvm_s390_pv_remove_old_asce (kvm); + + page =alloc_pages (GFP_KERNEL_ACCOUNT ,CRST_ALLOC_ORDER ); +if (!page) +return -ENOMEM ; +list_add (&page->lru, &kvm->arch.gmap->crst_list); + + table =page_to_virt (page); +memcpy (table, kvm->arch.gmap->table,1UL << (CRST_ALLOC_ORDER +PAGE_SHIFT ));
Don't we want to memcpy first and then add it to the list? The gmap is still active per-se so I think we want to take the guest_table_lock for the list_add here.
arch/s390/kvm/pv.c static int kvm_s390_pv_alloc_vm(struct kvm *kvm) · line 202
Quoted 5 lines
arch/s390/kvm/pv.c +5
-0+ + asce = (kvm->arch.gmap->asce & ~PAGE_MASK ) |__pa (table); +WRITE_ONCE (kvm->arch.gmap->asce, asce); +WRITE_ONCE (kvm->mm->context.gmap_asce, asce); +WRITE_ONCE (kvm->arch.gmap->table, table);
If I remember correctly those won't need locks but I'm not 100% sure so please have a look at that.
arch/s390/kvm/pv.c static int kvm_s390_pv_alloc_vm(struct kvm *kvm) · line 205
Quoted 3 lines
arch/s390/kvm/pv.c +3
-0+ +return 0 ; +}
That should both be in gmap.c
Quoted 18 lines
arch/s390/kvm/pv.c +4
-1+/* this should not fail, but if it does, we must not free the donated memory */ int kvm_s390_pv_deinit_vm (struct kvm *kvm,u16 *rc,u16 *rrc) {@@ -169,9 +218,11 @@ int kvm_s390_pv_deinit_vm(struct kvm *kvm, u16 *rc, u16 *rrc)atomic_set (&kvm->mm->context.is_protected,0 );KVM_UV_EVENT (kvm,3 ,"PROTVIRT DESTROY VM: rc %x rrc %x" , *rc, *rrc);WARN_ONCE (cc,"protvirt destroy vm failed rc %x rrc %x" , *rc, *rrc); -/* Inteded memory leak on "impossible" error */ +/* Intended memory leak on "impossible" error */ if (!cc)kvm_s390_pv_dealloc_vm (kvm); +else +kvm_s390_pv_replace_asce (kvm);return cc ? -EIO :0 ; }
Claudio Imbrenda
maintainerRe: [PATCH v2 02/13] KVM: s390: pv: leak the ASCE page when destroy fails
Quoted 70 lines · On Thu, 29 Jul 2021 12:41:05 +0200 Janosch Frank <[email protected]> wrote:
Quoted 69 lines · Claudio Imbrenda
When a protected VM is created, the topmost level of page tables of its ASCE is marked by the Ultravisor; any attempt to use that memory for protected virtualization will result in failure.
Only a successful Destroy Configuration UVC will remove the marking.
When the Destroy Configuration UVC fails, the topmost level of page tables of the VM does not get its marking cleared; to avoid issues it must not be used again.
Since the page becomes in practice unusable, we set it aside and leak it.
Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/kvm/pv.c | 53+++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 52 insertions(+), 1 deletion(-)
arch/s390/kvm/pv.c +2
-0@@ -155,6 +155,55 @@ static int kvm_s390_pv_alloc_vm(struct kvm *kvm) return -ENOMEM; } +/* +* Remove the topmost level of page tables from the list of page tables of
+ * the gmap.
+ * This means that it will not be freed when the VM is torn down, and needs
+ * to be handled separately by the caller, unless an intentional leak is
+ * intended.
+ */arch/s390/kvm/pv.c +13
-0+static void kvm_s390_pv_remove_old_asce (struct kvm *kvm) +{ +struct page *old; + + old =virt_to_page (kvm->arch.gmap->table); +list_del (&old->lru); +/* in case the ASCE needs to be "removed" multiple times */ +INIT_LIST_HEAD (&old->lru); +} + +/* +* Try to replace the current ASCE with another equivalent one. +* If the allocation of the new top level page table fails, the ASCE is not
+ * replaced.
+ * In any case, the old ASCE is removed from the list, therefore the caller
+ * has to make sure to save a pointer to it beforehands, unless an
+ * intentional leak is intended.
+ */arch/s390/kvm/pv.c +12
-0+static int kvm_s390_pv_replace_asce (struct kvm *kvm) +{ +unsigned long asce; +struct page *page; +void *table; + +kvm_s390_pv_remove_old_asce (kvm); + + page =alloc_pages (GFP_KERNEL_ACCOUNT ,CRST_ALLOC_ORDER ); +if (!page) +return -ENOMEM ; +list_add (&page->lru, &kvm->arch.gmap->crst_list);
Quoted 8 lines
Quoted 4 lines
arch/s390/kvm/pv.c +3
-0+ + table =page_to_virt (page); + memcpy(table, kvm->arch.gmap->table, 1UL <<(
CRST_ALLOC_ORDER+PAGE_SHIFT));Don't we want to memcpy first and then add it to the list? The gmap is still active per-se so I think we want to take the
guest_table_lockfor thelist_addhere.
doesn't really make a difference, it is not actually used until a few lines later
also, the list is only ever touched here, during guest creation and destruction; IIRC in all those cases we hold kvm->lock
Quoted 8 lines
arch/s390/kvm/pv.c static int kvm_s390_pv_alloc_vm(struct kvm *kvm) · line 202
Quoted 5 lines
arch/s390/kvm/pv.c +5
-0+ + asce = (kvm->arch.gmap->asce & ~PAGE_MASK ) |__pa (table); +WRITE_ONCE (kvm->arch.gmap->asce, asce); +WRITE_ONCE (kvm->mm->context.gmap_asce, asce); +WRITE_ONCE (kvm->arch.gmap->table, table);If I remember correctly those won't need locks but I'm not 100% sure so please have a look at that.
it should not need locks, the VM is in use, so it can't disappear under our feet.
Quoted 5 lines
arch/s390/kvm/pv.c static int kvm_s390_pv_alloc_vm(struct kvm *kvm) · line 205
Quoted 3 lines
arch/s390/kvm/pv.c +3
-0+ +return 0 ; +}That should both be in
gmap.c
why?
Quoted 20 lines
Quoted 19 lines
+
/* this should not fail, but if it does, we must not free the
donated memory */ intkvm_s390_pv_deinit_vm(struct kvm *kvm, u16 *rc, u16 *rrc){arch/s390/kvm/pv.c +3
-1@@ -169,9 +218,11 @@ int kvm_s390_pv_deinit_vm(struct kvm *kvm, u16 *rc, u16 *rrc) atomic_set(&kvm->mm->context.is_protected, 0);KVM_UV_EVENT (kvm,3 ,"PROTVIRT DESTROY VM: rc %x rrc %x" , *rc, *rrc); WARN_ONCE(cc, "protvirt destroy vm failed rc %x rrc %x", *rc, *rrc); - /* Inteded memory leak on "impossible" error */ + /* Intended memory leak on "impossible" error */ if (!cc) kvm_s390_pv_dealloc_vm(kvm); + else +kvm_s390_pv_replace_asce (kvm );return cc ? -EIO :0 ; }
Janosch Frank
maintainerRe: [PATCH v2 02/13] KVM: s390: pv: leak the ASCE page when destroy fails
Quoted 82 lines · Claudio Imbrenda
Quoted 70 lines · On Thu, 29 Jul 2021 12:41:05 +0200 Janosch Frank <[email protected]> wrote:
Quoted 69 lines · Claudio Imbrenda
When a protected VM is created, the topmost level of page tables of its ASCE is marked by the Ultravisor; any attempt to use that memory for protected virtualization will result in failure.
Only a successful Destroy Configuration UVC will remove the marking.
When the Destroy Configuration UVC fails, the topmost level of page tables of the VM does not get its marking cleared; to avoid issues it must not be used again.
Since the page becomes in practice unusable, we set it aside and leak it.
Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/kvm/pv.c | 53+++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 52 insertions(+), 1 deletion(-)
arch/s390/kvm/pv.c +2
-0@@ -155,6 +155,55 @@ static int kvm_s390_pv_alloc_vm(struct kvm *kvm) return -ENOMEM; } +/* +* Remove the topmost level of page tables from the list of page tables of
+ * the gmap.
+ * This means that it will not be freed when the VM is torn down, and needs
+ * to be handled separately by the caller, unless an intentional leak is
+ * intended.
+ */arch/s390/kvm/pv.c +13
-0+static void kvm_s390_pv_remove_old_asce (struct kvm *kvm) +{ +struct page *old; + + old =virt_to_page (kvm->arch.gmap->table); +list_del (&old->lru); +/* in case the ASCE needs to be "removed" multiple times */ +INIT_LIST_HEAD (&old->lru); +} + +/* +* Try to replace the current ASCE with another equivalent one. +* If the allocation of the new top level page table fails, the ASCE is not
+ * replaced.
+ * In any case, the old ASCE is removed from the list, therefore the caller
+ * has to make sure to save a pointer to it beforehands, unless an
+ * intentional leak is intended.
+ */arch/s390/kvm/pv.c +12
-0+static int kvm_s390_pv_replace_asce (struct kvm *kvm) +{ +unsigned long asce; +struct page *page; +void *table; + +kvm_s390_pv_remove_old_asce (kvm); + + page =alloc_pages (GFP_KERNEL_ACCOUNT ,CRST_ALLOC_ORDER ); +if (!page) +return -ENOMEM ; +list_add (&page->lru, &kvm->arch.gmap->crst_list);Quoted 8 lines
Quoted 4 lines
arch/s390/kvm/pv.c +3
-0+ + table =page_to_virt (page); + memcpy(table, kvm->arch.gmap->table, 1UL <<(
CRST_ALLOC_ORDER+PAGE_SHIFT));Don't we want to memcpy first and then add it to the list? The gmap is still active per-se so I think we want to take the
guest_table_lockfor thelist_addhere.
Quoted 2 lines
doesn't really make a difference, it is not actually used until a few lines later
Still when you touch the patch, then just move that around please :)
Quoted 3 lines
also, the list is only ever touched here, during guest creation and destruction; IIRC in all those cases we hold
kvm->lock
The crst_list is also used in gmap_alloc_table() coming from __gmap_link() when touching the top level entries. To be fair there shouldn't be asynchronous things that trigger a fault/link but we have valid references to the gmap so at the very least I want a comment why this is totally fine like we have for gmap_free().
Hrm, I'll need to brood over this a bit more, it has been too long.
Quoted 13 lines
Quoted 9 lines
arch/s390/kvm/pv.c static int kvm_s390_pv_alloc_vm(struct kvm *kvm) · line 202
Quoted 6 lines
arch/s390/kvm/pv.c +5
-0+ + asce = (kvm->arch.gmap->asce & ~PAGE_MASK ) |__pa (table); +WRITE_ONCE (kvm->arch.gmap->asce, asce); +WRITE_ONCE (kvm->mm->context.gmap_asce, asce); +WRITE_ONCE (kvm->arch.gmap->table, table);If I remember correctly those won't need locks but I'm not 100% sure so please have a look at that.
it should not need locks, the VM is in use, so it can't disappear under our feet.
Quoted 7 lines
Quoted 5 lines
arch/s390/kvm/pv.c static int kvm_s390_pv_alloc_vm(struct kvm *kvm) · line 205
Quoted 3 lines
arch/s390/kvm/pv.c +3
-0+ +return 0 ; +}That should both be in
gmap.cwhy?
Because we do gmap stuff and I don't want to pollute pv.c with memory things if we don't have to.
Btw. s390_reset_acc() is also in gmap.c
Quoted 22 lines
Quoted 21 lines
Quoted 20 lines
+
/* this should not fail, but if it does, we must not free the
donated memory */ intkvm_s390_pv_deinit_vm(struct kvm *kvm, u16 *rc, u16 *rrc){arch/s390/kvm/pv.c +3
-1@@ -169,9 +218,11 @@ int kvm_s390_pv_deinit_vm(struct kvm *kvm, u16 *rc, u16 *rrc) atomic_set(&kvm->mm->context.is_protected, 0);KVM_UV_EVENT (kvm,3 ,"PROTVIRT DESTROY VM: rc %x rrc %x" , *rc, *rrc); WARN_ONCE(cc, "protvirt destroy vm failed rc %x rrc %x", *rc, *rrc); - /* Inteded memory leak on "impossible" error */ + /* Intended memory leak on "impossible" error */ if (!cc) kvm_s390_pv_dealloc_vm(kvm); + else +kvm_s390_pv_replace_asce (kvm );return cc ? -EIO :0 ; }
Claudio Imbrenda
maintainerPATCH v2 3/13 KVM: s390: pv: properly handle page flags for protected guests
In reply to Claudio ImbrendaIntroduce variants of the convert and destroy page functions that also clear the PG_arch_1 bit used to mark them as secure pages.
These new functions can only be called on pages for which a reference is already being held.
--- arch/s390/include/asm/pgtable.h | 9 ++++++--- arch/s390/include/asm/uv.h | 10 ++++++++-- arch/s390/kernel/uv.c | 34 ++++++++++++++++++++++++++++++++- arch/s390/mm/gmap.c | 4 +++- 4 files changed, 50 insertions(+), 7 deletions(-)
arch/s390/include/asm/pgtable.h +6 -3
@@ -1074,8 +1074,9 @@ static inline pte_t ptep_get_and_clear(struct mm_struct *mm,pte_t res; res =ptep_xchg_lazy (mm, addr, ptep,__pte (_PAGE_INVALID)); +/* At this point the reference through the mapping is still present */ if (mm_is_protected (mm) &&pte_present (res)) -uv_convert_from_secure (pte_val (res) &PAGE_MASK ); +uv_convert_owned_from_secure (pte_val (res) &PAGE_MASK );return res; }
@@ -1091,8 +1092,9 @@ static inline pte_t ptep_clear_flush(struct vm_area_struct *vma,pte_t res; res =ptep_xchg_direct (vma->vm_mm, addr, ptep,__pte (_PAGE_INVALID)); +/* At this point the reference through the mapping is still present */ if (mm_is_protected (vma->vm_mm) &&pte_present (res)) -uv_convert_from_secure (pte_val (res) &PAGE_MASK ); +uv_convert_owned_from_secure (pte_val (res) &PAGE_MASK );return res; }
@@ -1116,8 +1118,9 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm, }else { res =ptep_xchg_lazy (mm, addr, ptep,__pte (_PAGE_INVALID)); } +/* At this point the reference through the mapping is still present */ if (mm_is_protected (mm) &&pte_present (res)) -uv_convert_from_secure (pte_val (res) &PAGE_MASK ); +uv_convert_owned_from_secure (pte_val (res) &PAGE_MASK );return res; }
arch/s390/include/asm/uv.h +8 -2
@@ -351,8 +351,9 @@ static inline int is_prot_virt_host(void) }int gmap_make_secure (struct gmap *gmap,unsigned long gaddr,void *uvcb); -int uv_destroy_page (unsigned long paddr); +int uv_destroy_owned_page (unsigned long paddr);int uv_convert_from_secure (unsigned long paddr); +int uv_convert_owned_from_secure (unsigned long paddr);int gmap_convert_to_secure (struct gmap *gmap,unsigned long gaddr);void setup_uv (void );
@@ -362,7 +363,7 @@ void adjust_to_uv_max(unsigned long *vmax);static inline void setup_uv (void ) {}static inline void adjust_to_uv_max (unsigned long *vmax) {} -static inline int uv_destroy_page (unsigned long paddr) +static inline int uv_destroy_owned_page (unsigned long paddr) {return 0 ; }
@@ -371,6 +372,11 @@ static inline int uv_convert_from_secure(unsigned long paddr) {return 0 ; } + +static inline int uv_convert_owned_from_secure (unsigned long paddr) +{ +return 0 ; +}#endif #if defined(CONFIG_PROTECTED_VIRTUALIZATION_GUEST ) ||IS_ENABLED (CONFIG_KVM )
arch/s390/kernel/uv.c +33 -1
@@ -115,7 +115,7 @@ static int uv_pin_shared(unsigned long paddr)* * @paddr: Absolute host address of page to be destroyed */ -int uv_destroy_page (unsigned long paddr) +static int uv_destroy_page (unsigned long paddr) {struct uv_cb_cfs uvcb = { .header.cmd =UVC_CMD_DESTR_SEC_STOR ,
@@ -135,6 +135,22 @@ int uv_destroy_page(unsigned long paddr)return 0 ; } +/* +* The caller must already hold a reference to the page +*/ +int uv_destroy_owned_page (unsigned long paddr) +{ +struct page *page =phys_to_page (paddr); +int rc; + +get_page (page); + rc =uv_destroy_page (paddr); +if (!rc) +clear_bit (PG_arch_1, &page->flags); +put_page (page); +return rc; +} +/* * Requests the Ultravisor to encrypt a guest page and make it * accessible to the host for paging (export).
@@ -154,6 +170,22 @@ int uv_convert_from_secure(unsigned long paddr)return 0 ; } +/* +* The caller must already hold a reference to the page +*/ +int uv_convert_owned_from_secure (unsigned long paddr) +{ +struct page *page =phys_to_page (paddr); +int rc; + +get_page (page); + rc =uv_convert_from_secure (paddr); +if (!rc) +clear_bit (PG_arch_1, &page->flags); +put_page (page); +return rc; +} +/* * Calculate the expected ref_count for a page that would otherwise have no * further pins. This was cribbed from similar functions in other places in
arch/s390/mm/gmap.c +3 -1
@@ -2678,8 +2678,10 @@ static int __s390_reset_acc(pte_t *ptep, unsigned long addr, {pte_t pte =READ_ONCE (*ptep); +/* There is a reference through the mapping */ if (pte_present (pte)) -WARN_ON_ONCE (uv_destroy_page (pte_val (pte) &PAGE_MASK )); +WARN_ON_ONCE (uv_destroy_owned_page (pte_val (pte) &PAGE_MASK )); +return 0 ; }
Signature · 1 line
2.31.1
Re: [PATCH v2 03/13] KVM: s390: pv: properly handle page flags for protected guests
Quoted 7 lines · Claudio Imbrenda
Introduce variants of the convert and destroy page functions that also clear the
PG_arch_1bit used to mark them as secure pages.These new functions can only be called on pages for which a reference is already being held.
Signed-off-by Claudio Imbrenda <[email protected]>
Quoted 156 lines
--- arch/s390/include/asm/pgtable.h | 9 ++++++--- arch/s390/include/asm/uv.h | 10 ++++++++-- arch/s390/kernel/uv.c | 34 ++++++++++++++++++++++++++++++++- arch/s390/mm/gmap.c | 4 +++- 4 files changed, 50 insertions(+), 7 deletions(-)arch/s390/include/asm/pgtable.h +6
-3@@ -1074,8 +1074,9 @@ static inline pte_t ptep_get_and_clear(struct mm_struct *mm,pte_t res; res =ptep_xchg_lazy (mm, addr, ptep,__pte (_PAGE_INVALID)); +/* At this point the reference through the mapping is still present */ if (mm_is_protected (mm) &&pte_present (res)) -uv_convert_from_secure (pte_val (res) &PAGE_MASK ); +uv_convert_owned_from_secure (pte_val (res) &PAGE_MASK );return res; }@@ -1091,8 +1092,9 @@ static inline pte_t ptep_clear_flush(struct vm_area_struct *vma,pte_t res; res =ptep_xchg_direct (vma->vm_mm, addr, ptep,__pte (_PAGE_INVALID)); +/* At this point the reference through the mapping is still present */ if (mm_is_protected (vma->vm_mm) &&pte_present (res)) -uv_convert_from_secure (pte_val (res) &PAGE_MASK ); +uv_convert_owned_from_secure (pte_val (res) &PAGE_MASK );return res; }@@ -1116,8 +1118,9 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm, }else { res =ptep_xchg_lazy (mm, addr, ptep,__pte (_PAGE_INVALID)); } +/* At this point the reference through the mapping is still present */ if (mm_is_protected (mm) &&pte_present (res)) -uv_convert_from_secure (pte_val (res) &PAGE_MASK ); +uv_convert_owned_from_secure (pte_val (res) &PAGE_MASK );return res; }arch/s390/include/asm/uv.h +8
-2@@ -351,8 +351,9 @@ static inline int is_prot_virt_host(void) }int gmap_make_secure (struct gmap *gmap,unsigned long gaddr,void *uvcb); -int uv_destroy_page (unsigned long paddr); +int uv_destroy_owned_page (unsigned long paddr);int uv_convert_from_secure (unsigned long paddr); +int uv_convert_owned_from_secure (unsigned long paddr);int gmap_convert_to_secure (struct gmap *gmap,unsigned long gaddr);void setup_uv (void );@@ -362,7 +363,7 @@ void adjust_to_uv_max(unsigned long *vmax);static inline void setup_uv (void ) {}static inline void adjust_to_uv_max (unsigned long *vmax) {} -static inline int uv_destroy_page (unsigned long paddr) +static inline int uv_destroy_owned_page (unsigned long paddr) {return 0 ; }@@ -371,6 +372,11 @@ static inline int uv_convert_from_secure(unsigned long paddr) {return 0 ; } + +static inline int uv_convert_owned_from_secure (unsigned long paddr) +{ +return 0 ; +}#endif #if defined(CONFIG_PROTECTED_VIRTUALIZATION_GUEST ) ||IS_ENABLED (CONFIG_KVM )arch/s390/kernel/uv.c +33
-1@@ -115,7 +115,7 @@ static int uv_pin_shared(unsigned long paddr)* * @paddr: Absolute host address of page to be destroyed */ -int uv_destroy_page (unsigned long paddr) +static int uv_destroy_page (unsigned long paddr) {struct uv_cb_cfs uvcb = { .header.cmd =UVC_CMD_DESTR_SEC_STOR ,@@ -135,6 +135,22 @@ int uv_destroy_page(unsigned long paddr)return 0 ; } +/* +* The caller must already hold a reference to the page +*/ +int uv_destroy_owned_page (unsigned long paddr) +{ +struct page *page =phys_to_page (paddr); +int rc; + +get_page (page); + rc =uv_destroy_page (paddr); +if (!rc) +clear_bit (PG_arch_1, &page->flags); +put_page (page); +return rc; +} +/* * Requests the Ultravisor to encrypt a guest page and make it * accessible to the host for paging (export). @@ -154,6 +170,22 @@ int uv_convert_from_secure(unsigned long paddr)return 0 ; } +/* +* The caller must already hold a reference to the page +*/ +int uv_convert_owned_from_secure (unsigned long paddr) +{ +struct page *page =phys_to_page (paddr); +int rc; + +get_page (page); + rc =uv_convert_from_secure (paddr); +if (!rc) +clear_bit (PG_arch_1, &page->flags); +put_page (page); +return rc; +} +/* * Calculate the expected ref_count for a page that would otherwise have no * further pins. This was cribbed from similar functions in other places in arch/s390/mm/gmap.c +3
-1@@ -2678,8 +2678,10 @@ static int __s390_reset_acc(pte_t *ptep, unsigned long addr, {pte_t pte =READ_ONCE (*ptep); +/* There is a reference through the mapping */ if (pte_present (pte)) -WARN_ON_ONCE (uv_destroy_page (pte_val (pte) &PAGE_MASK )); +WARN_ON_ONCE (uv_destroy_owned_page (pte_val (pte) &PAGE_MASK )); +return 0 ; }
Claudio Imbrenda
maintainerPATCH v2 4/13 KVM: s390: pv: handle secure storage violations for protected guests
In reply to Claudio ImbrendaWith upcoming patches, protected guests will be able to trigger secure storage violations in normal operation.
This patch adds handling of secure storage violations for protected guests.
Pages that trigger the exception will be made non-secure before attempting to use them again.
--- arch/s390/include/asm/uv.h | 1 + arch/s390/kernel/uv.c | 43 ++++++++++++++++++++++++++++++++++++++ arch/s390/mm/fault.c | 10 +++++++++ 3 files changed, 54 insertions(+)
arch/s390/include/asm/uv.h +1 -0
@@ -351,6 +351,7 @@ static inline int is_prot_virt_host(void) }int gmap_make_secure (struct gmap *gmap,unsigned long gaddr,void *uvcb); +int gmap_destroy_page (struct gmap *gmap,unsigned long gaddr);int uv_destroy_owned_page (unsigned long paddr);int uv_convert_from_secure (unsigned long paddr);int uv_convert_owned_from_secure (unsigned long paddr);
arch/s390/kernel/uv.c +43 -0
49 lines not on this page · raw message
arch/s390/mm/fault.c +10 -0
@@ -850,6 +850,16 @@ NOKPROBE_SYMBOL(do_non_secure_storage_access);void do_secure_storage_violation (struct pt_regs *regs) { +unsigned long gaddr = regs->int_parm_long & __FAIL_ADDR_MASK; +struct gmap *gmap = (struct gmap *)S390_lowcore.gmap; + +/* +* If the VM has been rebooted, its address space might still contain +* secure pages from the previous boot. +* Clear the page so it can be reused. +*/ +if (!gmap_destroy_page (gmap, gaddr)) +return ;/* * Either KVM messed up the secure guest mapping or the same * page is mapped into multiple secure guests.
Signature · 1 line
2.31.1
Claudio Imbrenda
maintainerPATCH v2 5/13 KVM: s390: pv: handle secure storage exceptions for normal guests
In reply to Claudio ImbrendaWith upcoming patches, normal guests might touch secure pages.
This patch extends the existing exception handler to convert the pages to non secure also when the exception is triggered by a normal guest.
This can happen for example when a secure guest reboots; the first stage of a secure guest is non secure, and in general a secure guest can reboot into non-secure mode.
If the secure memory of the previous boot has not been cleared up completely yet, a non-secure guest might touch secure memory, which will need to be handled properly.
--- arch/s390/mm/fault.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
arch/s390/mm/fault.c +11 -1
@@ -767,6 +767,7 @@ void do_secure_storage_access(struct pt_regs *regs)struct vm_area_struct *vma;struct mm_struct *mm;struct page *page; +struct gmap *gmap;int rc;/*
@@ -796,6 +797,16 @@ void do_secure_storage_access(struct pt_regs *regs) }switch (get_fault_type (regs)) { +case GMAP_FAULT : + gmap = (struct gmap *)S390_lowcore.gmap; +/* +* Very unlikely, but if it happens, simply try again. +* The next attempt will trigger a different exception. +*/ + addr =__gmap_translate (gmap, addr); +if (addr == -EFAULT ) +break ; + fallthrough;case USER_FAULT : mm = current->mm;mmap_read_lock (mm);
@@ -824,7 +835,6 @@ void do_secure_storage_access(struct pt_regs *regs)if (rc)BUG ();break ; -case GMAP_FAULT :default :do_fault_error (regs,VM_READ |VM_WRITE ,VM_FAULT_BADMAP );WARN_ON_ONCE (1 );
Signature · 1 line
2.31.1
Janosch Frank
maintainerRe: [PATCH v2 05/13] KVM: s390: pv: handle secure storage exceptions for normal guests
Quoted 31 lines · Claudio Imbrenda
With upcoming patches, normal guests might touch secure pages.
This patch extends the existing exception handler to convert the pages to non secure also when the exception is triggered by a normal guest.
This can happen for example when a secure guest reboots; the first stage of a secure guest is non secure, and in general a secure guest can reboot into non-secure mode.
If the secure memory of the previous boot has not been cleared up completely yet, a non-secure guest might touch secure memory, which will need to be handled properly.
Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/mm/fault.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)arch/s390/mm/fault.c +1
-0@@ -767,6 +767,7 @@ void do_secure_storage_access(struct pt_regs *regs)struct vm_area_struct *vma;struct mm_struct *mm;struct page *page; +struct gmap *gmap;int rc;/*
arch/s390/mm/fault.c void do_secure_storage_access(struct pt_regs *regs) · line 805
Quoted 9 lines
arch/s390/mm/fault.c +6
-0@@ -796,6 +797,16 @@ void do_secure_storage_access(struct pt_regs *regs) }switch (get_fault_type (regs)) { +case GMAP_FAULT : + gmap = (struct gmap *)S390_lowcore.gmap; +/* +* Very unlikely, but if it happens, simply try again. +* The next attempt will trigger a different exception. +*/
If we keep this the way it currently is then the comment needs to go to the EFAULT check since it makes no sense above the gmap_translate().
arch/s390/mm/fault.c void do_secure_storage_access(struct pt_regs *regs) · line 806
Quoted 1 line
arch/s390/mm/fault.c +1
-0+ addr =__gmap_translate (gmap, addr);
So we had a valid gmap PTE to end up here where the guest touched a secure page and triggered the exception. But we suddenly can't translate the gaddr to a vmaddr because the gmap tracking doesn't have an entry for the address.
My first instinct is to SIGSEGV the process since I can't come up with a way out of this situation except for the process to map this back in. The only reason I can think of that it was removed from the mapping is malicious intent or a bug.
I think this is needs a VM_FAULT_BADMAP and a do_fault_error() call.
Quoted 15 lines
arch/s390/mm/fault.c +3
-1+if (addr == -EFAULT ) +break ; + fallthrough;case USER_FAULT : mm = current->mm;mmap_read_lock (mm);@@ -824,7 +835,6 @@ void do_secure_storage_access(struct pt_regs *regs)if (rc)BUG ();break ; -case GMAP_FAULT :default :do_fault_error (regs,VM_READ |VM_WRITE ,VM_FAULT_BADMAP );WARN_ON_ONCE (1 );
Claudio Imbrenda
maintainerRe: [PATCH v2 05/13] KVM: s390: pv: handle secure storage exceptions for normal guests
Quoted 56 lines · On Thu, 29 Jul 2021 14:17:11 +0200 Janosch Frank <[email protected]> wrote:
Quoted 42 lines · Claudio Imbrenda
With upcoming patches, normal guests might touch secure pages.
This patch extends the existing exception handler to convert the pages to non secure also when the exception is triggered by a normal guest.
This can happen for example when a secure guest reboots; the first stage of a secure guest is non secure, and in general a secure guest can reboot into non-secure mode.
If the secure memory of the previous boot has not been cleared up completely yet, a non-secure guest might touch secure memory, which will need to be handled properly.
Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/mm/fault.c | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)arch/s390/mm/fault.c +5
-0@@ -767,6 +767,7 @@ void do_secure_storage_access(struct pt_regs *regs) struct vm_area_struct *vma;struct mm_struct *mm;struct page *page; +struct gmap *gmap;int rc;/* @@ -796,6 +797,16 @@ void do_secure_storage_access(struct pt_regs *regs) }switch (get_fault_type (regs)) { +case GMAP_FAULT : + gmap = (struct gmap *)S390_lowcore.gmap; +/* +* Very unlikely, but if it happens, simply try again.
arch/s390/mm/fault.c +1
-0+* The next attempt will trigger a different exception.
arch/s390/mm/fault.c void do_secure_storage_access(struct pt_regs *regs) · line 805
Quoted 1 line
arch/s390/mm/fault.c +1
-0+*/ If we keep this the way it currently is then the comment needs to go to the EFAULT check since it makes no sense above the
gmap_translate().arch/s390/mm/fault.c void do_secure_storage_access(struct pt_regs *regs) · line 806
Quoted 1 line
arch/s390/mm/fault.c +1
-0+ addr =__gmap_translate (gmap, addr);So we had a valid gmap PTE to end up here where the guest touched a secure page and triggered the exception. But we suddenly can't translate the gaddr to a vmaddr because the gmap tracking doesn't have an entry for the address.
Quoted 6 lines
My first instinct is to SIGSEGV the process since I can't come up with a way out of this situation except for the process to map this back in. The only reason I can think of that it was removed from the mapping is malicious intent or a bug.
I think this is needs a
VM_FAULT_BADMAPand ado_fault_error()call.
fair enough, the next version will have that
Quoted 16 lines
Quoted 15 lines
arch/s390/mm/fault.c +3
-1+if (addr == -EFAULT ) +break ; + fallthrough;case USER_FAULT : mm = current->mm;mmap_read_lock (mm);@@ -824,7 +835,6 @@ void do_secure_storage_access(struct pt_regs *regs) if (rc)BUG ();break ; -case GMAP_FAULT :default :do_fault_error (regs,VM_READ |VM_WRITE ,VM_FAULT_BADMAP );WARN_ON_ONCE (1 );
Claudio Imbrenda
maintainerPATCH v2 6/13 KVM: s390: pv: refactor s390_reset_acc
In reply to Claudio ImbrendaRefactor s390_reset_acc so that its pieces can be reused in upcoming patches. The users parameter for s390_destroy_range will be needed in upcoming patches.
We don't want to hold all the locks used in a walk_page_range for too long, and the destroy page UVC does take some time to complete. Therefore we quickly gather the pages to destroy, and then destroy them without holding all the locks.
--- arch/s390/include/asm/gmap.h | 5 +- arch/s390/kvm/pv.c | 12 ++++- arch/s390/mm/gmap.c | 88 ++++++++++++++++++++++++------------ 3 files changed, 73 insertions(+), 32 deletions(-)
arch/s390/include/asm/gmap.h +4 -1
@@ -147,5 +147,8 @@ int gmap_mprotect_notify(struct gmap *, unsigned long start,void gmap_sync_dirty_log_pmd (struct gmap *gmap,unsigned long dirty_bitmap[4 ],unsigned long gaddr,unsigned long vmaddr);int gmap_mark_unmergeable (void ); -void s390_reset_acc (struct mm_struct *mm); +void s390_uv_destroy_range (struct mm_struct *mm,unsigned int users, +unsigned long start,unsigned long end); +void s390_uv_destroy_pfns (unsigned long count,unsigned long *pfns); +#endif /* _ASM_S390_GMAP_H */
arch/s390/kvm/pv.c +10 -2
@@ -12,6 +12,8 @@#include <asm/gmap.h> #include <asm/uv.h> #include <asm/mman.h> +#include <linux/pagewalk.h> +#include <linux/sched/mm.h> #include "kvm-s390.h" int kvm_s390_pv_destroy_cpu (struct kvm_vcpu *vcpu,u16 *rc,u16 *rrc)
@@ -209,8 +211,14 @@ int kvm_s390_pv_deinit_vm(struct kvm *kvm, u16 *rc, u16 *rrc) {int cc; -/* make all pages accessible before destroying the guest */ -s390_reset_acc (kvm->mm); +/* +* if the mm still has a mapping, make all its pages accessible +* before destroying the guest +*/ +if (mmget_not_zero (kvm->mm)) { +s390_uv_destroy_range (kvm->mm,0 ,0 ,TASK_SIZE ); +mmput (kvm->mm); + } cc =uv_cmd_nodata (kvm_s390_pv_get_handle (kvm),UVC_CMD_DESTROY_SEC_CONF , rc, rrc);
arch/s390/mm/gmap.c +59 -29
100 lines not on this page · raw message
Signature · 1 line
2.31.1
Claudio Imbrenda
maintainerPATCH v2 7/13 KVM: s390: pv: usage counter instead of flag
In reply to Claudio ImbrendaUse the is_protected field as a counter instead of a flag. This will be used in upcoming patches.
Increment the counter when a secure configuration is created, and decrement it when it is destroyed. Previously the flag was set when the set secure parameters UVC was performed.
--- arch/s390/kvm/pv.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
arch/s390/kvm/pv.c +7 -5
@@ -223,7 +223,8 @@ int kvm_s390_pv_deinit_vm(struct kvm *kvm, u16 *rc, u16 *rrc) cc =uv_cmd_nodata (kvm_s390_pv_get_handle (kvm),UVC_CMD_DESTROY_SEC_CONF , rc, rrc);WRITE_ONCE (kvm->arch.gmap->guest_handle,0 ); -atomic_set (&kvm->mm->context.is_protected,0 ); +if (!cc) +atomic_dec (&kvm->mm->context.is_protected);KVM_UV_EVENT (kvm,3 ,"PROTVIRT DESTROY VM: rc %x rrc %x" , *rc, *rrc);WARN_ONCE (cc,"protvirt destroy vm failed rc %x rrc %x" , *rc, *rrc);/* Intended memory leak on "impossible" error */
@@ -264,11 +265,14 @@ int kvm_s390_pv_init_vm(struct kvm *kvm, u16 *rc, u16 *rrc)/* Outputs */ kvm->arch.pv.handle = uvcb.guest_handle; +atomic_inc (&kvm->mm->context.is_protected);if (cc) { -if (uvcb.header.rc &UVC_RC_NEED_DESTROY ) +if (uvcb.header.rc &UVC_RC_NEED_DESTROY ) {kvm_s390_pv_deinit_vm (kvm, &dummy, &dummy); -else + }else { +atomic_dec (&kvm->mm->context.is_protected);kvm_s390_pv_dealloc_vm (kvm); + }return -EIO ; } kvm->arch.gmap->guest_handle = uvcb.guest_handle;
@@ -291,8 +295,6 @@ int kvm_s390_pv_set_sec_parms(struct kvm *kvm, void *hdr, u64 length, u16 *rc, *rrc = uvcb.header.rrc;KVM_UV_EVENT (kvm,3 ,"PROTVIRT VM SET PARMS: rc %x rrc %x" , *rc, *rrc); -if (!cc) -atomic_set (&kvm->mm->context.is_protected,1 );return cc ? -EINVAL :0 ; }
Signature · 1 line
2.31.1
Claudio Imbrenda
maintainerPATCH v2 8/13 KVM: s390: pv: add export before import
In reply to Claudio ImbrendaDue to upcoming changes, it will be possible to temporarily have multiple protected VMs in the same address space, although only one will be actually active.
In that scenario, it is necessary to perform an export of every page that is to be imported, since the hardware does not allow a page belonging to a protected guest to be imported into a different protected guest.
This also applies to pages that are shared, and thus accessible by the host.
--- arch/s390/kernel/uv.c | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/s390/kernel/uv.c +8 -0
@@ -241,6 +241,12 @@ static int make_secure_pte(pte_t *ptep, unsigned long addr,return rc; } +static bool should_export_before_import (struct uv_cb_header *uvcb,struct mm_struct *mm) +{ +return uvcb->cmd !=UVC_CMD_UNPIN_PAGE_SHARED && +atomic_read (&mm->context.is_protected) >1 ; +} +/* * Requests the Ultravisor to make a page accessible to a guest. * If it's brought in the first time, it will be cleared. If
@@ -284,6 +290,8 @@ int gmap_make_secure(struct gmap *gmap, unsigned long gaddr, void *uvcb)lock_page (page); ptep =get_locked_pte (gmap->mm, uaddr, &ptelock); +if (should_export_before_import (uvcb, gmap->mm)) +uv_convert_from_secure (page_to_phys (page)); rc =make_secure_pte (ptep, uaddr, page, uvcb);pte_unmap_unlock (ptep, ptelock);unlock_page (page);
Signature · 1 line
2.31.1
Claudio Imbrenda
maintainerPATCH v2 9/13 KVM: s390: pv: lazy destroy for reboot
In reply to Claudio ImbrendaUntil now, destroying a protected guest was an entirely synchronous operation that could potentially take a very long time, depending on the size of the guest, due to the time needed to clean up the address space from protected pages.
This patch implements a lazy destroy mechanism, that allows a protected guest to reboot significantly faster than previously.
This is achieved by clearing the pages of the old guest in background. In case of reboot, the new guest will be able to run in the same address space almost immediately.
The old protected guest is then only destroyed when all of its memory has been destroyed or otherwise made non protected.
--- arch/s390/kvm/kvm-s390.c | 6 +- arch/s390/kvm/kvm-s390.h | 2 +- arch/s390/kvm/pv.c | 129 ++++++++++++++++++++++++++++++++++++++- 3 files changed, 131 insertions(+), 6 deletions(-)
arch/s390/kvm/kvm-s390.c +3 -3
@@ -2281,7 +2281,7 @@ static int kvm_s390_handle_pv(struct kvm *kvm, struct kvm_pv_cmd *cmd) r =kvm_s390_cpus_to_pv (kvm, &cmd->rc, &cmd->rrc);if (r) -kvm_s390_pv_deinit_vm (kvm, &dummy, &dummy); +kvm_s390_pv_deinit_vm_deferred (kvm, &dummy, &dummy);/* we need to block service interrupts from now on */ set_bit (IRQ_PEND_EXT_SERVICE , &kvm->arch.float_int.masked_irqs);
@@ -2300,7 +2300,7 @@ static int kvm_s390_handle_pv(struct kvm *kvm, struct kvm_pv_cmd *cmd)*/ if (r)break ; - r =kvm_s390_pv_deinit_vm (kvm, &cmd->rc, &cmd->rrc); + r =kvm_s390_pv_deinit_vm_deferred (kvm, &cmd->rc, &cmd->rrc);/* no need to block service interrupts any more */ clear_bit (IRQ_PEND_EXT_SERVICE , &kvm->arch.float_int.masked_irqs);
@@ -2829,7 +2829,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)* complaining we do not use kvm_s390_pv_is_protected. */ if (kvm_s390_pv_get_handle (kvm)) -kvm_s390_pv_deinit_vm (kvm, &rc, &rrc); +kvm_s390_pv_deinit_vm_deferred (kvm, &rc, &rrc);debug_unregister (kvm->arch.dbf);free_page ((unsigned long )kvm->arch.sie_page2);if (!kvm_is_ucontrol (kvm))
arch/s390/kvm/kvm-s390.h +1 -1
@@ -211,7 +211,7 @@ static inline int kvm_s390_user_cpu_state_ctrl(struct kvm *kvm)/* implemented in pv.c */ int kvm_s390_pv_destroy_cpu (struct kvm_vcpu *vcpu,u16 *rc,u16 *rrc);int kvm_s390_pv_create_cpu (struct kvm_vcpu *vcpu,u16 *rc,u16 *rrc); -int kvm_s390_pv_deinit_vm (struct kvm *kvm,u16 *rc,u16 *rrc); +int kvm_s390_pv_deinit_vm_deferred (struct kvm *kvm,u16 *rc,u16 *rrc);int kvm_s390_pv_init_vm (struct kvm *kvm,u16 *rc,u16 *rrc);int kvm_s390_pv_set_sec_parms (struct kvm *kvm,void *hdr,u64 length,u16 *rc,u16 *rrc);
arch/s390/kvm/pv.c +127 -2
155 lines not on this page · raw message
Signature · 1 line
2.31.1
Claudio Imbrenda
maintainerPATCH v2 10/13 KVM: s390: pv: extend lazy destroy to handle shutdown
In reply to Claudio ImbrendaExtend the lazy destroy infrastructure to handle almost all kinds of exit of the userspace process. The only case not handled is if the process is killed by the OOM, in which case the old behaviour will still be in effect.
Add the uv_destroy_page_lazy function to set aside pages when unmapping them during mm teardown; the pages will be processed and freed when the protected VM is torn down.
--- arch/s390/include/asm/mmu.h | 3 ++ arch/s390/include/asm/mmu_context.h | 2 ++ arch/s390/include/asm/pgtable.h | 9 ++++-- arch/s390/include/asm/uv.h | 15 +++++++++ arch/s390/kernel/uv.c | 47 +++++++++++++++++++++++++++++ arch/s390/kvm/pv.c | 41 ++++++++++++++++++------- 6 files changed, 104 insertions(+), 13 deletions(-)
arch/s390/include/asm/mmu.h +3 -0
@@ -18,6 +18,7 @@ typedef struct {unsigned long vdso_base;/* The mmu context belongs to a secure guest. */ atomic_t is_protected; +struct list_head deferred_list;/* * The following bitfields need a down_write on the mm * semaphore when they are written to. As they are only
@@ -34,6 +35,8 @@ typedef struct { unsigned int uses_cmm:1;/* The gmaps associated with this context are allowed to use huge pages. */ unsigned int allow_gmap_hpage_1m:1; +/* The mmu context should be destroyed synchronously */ + unsigned int pv_sync_destroy:1; } mm_context_t;#define INIT_MM_CONTEXT (name) \
arch/s390/include/asm/mmu_context.h +2 -0
@@ -27,8 +27,10 @@ static inline int init_new_context(struct task_struct *tsk,cpumask_clear (&mm->context.cpu_attach_mask);atomic_set (&mm->context.flush_count,0 );atomic_set (&mm->context.is_protected,0 ); + mm->context.pv_sync_destroy =0 ; mm->context.gmap_asce =0 ; mm->context.flush_mm =0 ; +INIT_LIST_HEAD (&mm->context.deferred_list);#ifdef CONFIG_PGSTE mm->context.alloc_pgste = page_table_allocate_pgste || test_thread_flag(TIF_PGSTE) ||
arch/s390/include/asm/pgtable.h +7 -2
@@ -1118,9 +1118,14 @@ static inline pte_t ptep_get_and_clear_full(struct mm_struct *mm, }else { res =ptep_xchg_lazy (mm, addr, ptep,__pte (_PAGE_INVALID)); } +/* At this point the reference through the mapping is still present */ -if (mm_is_protected (mm) &&pte_present (res)) -uv_convert_owned_from_secure (pte_val (res) &PAGE_MASK ); +if (pte_present (res) &&mm_is_protected (mm)) { +if (full && !mm->context.pv_sync_destroy) +uv_destroy_page_lazy (mm,pte_val (res) &PAGE_MASK ); +else +uv_convert_owned_from_secure (pte_val (res) &PAGE_MASK ); + }return res; }
arch/s390/include/asm/uv.h +15 -0
@@ -345,6 +345,15 @@ static inline int uv_remove_shared(unsigned long addr) { return 0; }#if IS_ENABLED (CONFIG_KVM )extern int prot_virt_host; +struct destroy_page_lazy { +struct list_head list; +unsigned short count; +unsigned long pfns[]; +}; + +/* This guarantees that up to PV_MAX_LAZY_COUNT can fit in a page */ +#define PV_MAX_LAZY_COUNT ((PAGE_SIZE - sizeof(struct destroy_page_lazy)) / sizeof(long)) +static inline int is_prot_virt_host (void ) {return prot_virt_host;
@@ -353,6 +362,7 @@ static inline int is_prot_virt_host(void)int gmap_make_secure (struct gmap *gmap,unsigned long gaddr,void *uvcb);int gmap_destroy_page (struct gmap *gmap,unsigned long gaddr);int uv_destroy_owned_page (unsigned long paddr); +int uv_destroy_page_lazy (struct mm_struct *mm,unsigned long paddr);int uv_convert_from_secure (unsigned long paddr);int uv_convert_owned_from_secure (unsigned long paddr);int gmap_convert_to_secure (struct gmap *gmap,unsigned long gaddr);
@@ -369,6 +379,11 @@ static inline int uv_destroy_owned_page(unsigned long paddr)return 0 ; } +static inline int uv_destroy_page_lazy (struct mm_struct *mm,unsigned long paddr) +{ +return 0 ; +} +static inline int uv_convert_from_secure (unsigned long paddr) {return 0 ;
arch/s390/kernel/uv.c +47 -0
53 lines not on this page · raw message
arch/s390/kvm/pv.c +30 -11
70 lines not on this page · raw message
Signature · 1 line
2.31.1
Claudio Imbrenda
maintainerPATCH v2 11/13 KVM: s390: pv: module parameter to fence lazy destroy
In reply to Claudio ImbrendaAdd the module parameter "lazy_destroy", to allow the lazy destroy mechanism to be switched off. This might be useful for debugging purposes.
The parameter is enabled by default.
--- arch/s390/kvm/pv.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
arch/s390/kvm/pv.c +13 -0
@@ -26,6 +26,10 @@ struct deferred_priv {unsigned long stor_base; }; +static int lazy_destroy =1 ; +module_param (lazy_destroy, int,0444 ); +MODULE_PARM_DESC (lazy_destroy,"Deferred destroy for protected guests" ); +int kvm_s390_pv_destroy_cpu (struct kvm_vcpu *vcpu,u16 *rc,u16 *rrc) {int cc =0 ;
@@ -361,6 +365,9 @@ int kvm_s390_pv_deinit_vm_deferred(struct kvm *kvm, u16 *rc, u16 *rrc) {struct deferred_priv *priv; +if (!lazy_destroy) +return kvm_s390_pv_deinit_vm_now (kvm, rc, rrc); + priv =kmalloc (sizeof (*priv),GFP_KERNEL | __GFP_ZERO);if (!priv)return kvm_s390_pv_deinit_vm_now (kvm, rc, rrc);
@@ -409,6 +416,12 @@ int kvm_s390_pv_init_vm(struct kvm *kvm, u16 *rc, u16 *rrc)/* Outputs */ kvm->arch.pv.handle = uvcb.guest_handle; +if (!lazy_destroy) { +mmap_write_lock (kvm->mm); + kvm->mm->context.pv_sync_destroy =1 ; +mmap_write_unlock (kvm->mm); + } +atomic_inc (&kvm->mm->context.is_protected);if (cc) {if (uvcb.header.rc &UVC_RC_NEED_DESTROY ) {
Signature · 1 line
2.31.1
Claudio Imbrenda
maintainerPATCH v2 12/13 KVM: s390: pv: add OOM notifier for lazy destroy
In reply to Claudio ImbrendaAdd a per-VM OOM notifier for lazy destroy.
When a protected VM is undergoing deferred teardown, register an OOM notifier. This allows an OOM situation to be handled by just waiting a little.
The background cleanup deferred destroy process will now keep a running tally of the amount of pages freed. The asynchronous OOM notifier will check the number of pages freed before and after waiting. The OOM notifier will wait 10ms, and then report the number of pages freed by the deferred destroy mechanism during that time.
If at least 1024 pages have already been freed in the current OOM situation, no action is taken by the OOM notifier and no wait is performed. This avoids excessive waiting times in case many VMs are being destroyed at the same time, once enough memory has been freed.
--- arch/s390/kvm/pv.c | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+)
arch/s390/kvm/pv.c +32 -0
@@ -15,8 +15,12 @@#include <linux/pagewalk.h> #include <linux/sched/mm.h> #include <linux/kthread.h> +#include <linux/delay.h> +#include <linux/oom.h> #include "kvm-s390.h" +#define KVM_S390_PV_LAZY_DESTROY_OOM_NOTIFY_PRIORITY 70 +struct deferred_priv {struct mm_struct *mm;bool has_mm;
@@ -24,6 +28,8 @@ struct deferred_priv {u64 handle;void *stor_var;unsigned long stor_base; +unsigned long n_pages_freed; +struct notifier_block oom_nb; };static int lazy_destroy =1 ;
@@ -249,6 +255,24 @@ static int kvm_s390_pv_deinit_vm_now(struct kvm *kvm, u16 *rc, u16 *rrc)return cc ? -EIO :0 ; } +static int kvm_s390_pv_oom_notify (struct notifier_block *nb, +unsigned long dummy,void *parm) +{ +unsigned long *freed = parm; +unsigned long free_before; +struct deferred_priv *p; + +if (*freed >1024 ) +return NOTIFY_OK ; + + p = container_of(nb, struct deferred_priv, oom_nb); + free_before =READ_ONCE (p->n_pages_freed); +msleep (20 ); + *freed +=READ_ONCE (p->n_pages_freed) - free_before; + +return NOTIFY_OK ; +} +static int kvm_s390_pv_destroy_vm_thread (void *priv) {struct destroy_page_lazy *lazy, *next;
@@ -256,12 +280,20 @@ static int kvm_s390_pv_destroy_vm_thread(void *priv)u16 rc, rrc;int r; + p->oom_nb.priority =KVM_S390_PV_LAZY_DESTROY_OOM_NOTIFY_PRIORITY ; + p->oom_nb.notifier_call = kvm_s390_pv_oom_notify; + r =register_oom_notifier (&p->oom_nb); +list_for_each_entry_safe (lazy, next, &p->mm->context.deferred_list, list) {list_del (&lazy->list);s390_uv_destroy_pfns (lazy->count, lazy->pfns); +WRITE_ONCE (p->n_pages_freed, p->n_pages_freed + lazy->count +1 );free_page (__pa (lazy)); } +if (!r) +unregister_oom_notifier (&p->oom_nb); +if (p->has_mm) {/* Clear all the pages as long as we are not the only users of the mm */ s390_uv_destroy_range (p->mm,1 ,0 ,TASK_SIZE_MAX );
Signature · 1 line
2.31.1
Claudio Imbrenda
maintainerPATCH v2 13/13 KVM: s390: pv: add support for UV feature bits
In reply to Claudio ImbrendaAdd support for Ultravisor feature bits, and take advantage of the functionality advertised to speed up the lazy destroy mechanism.
--- arch/s390/kernel/uv.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/s390/kernel/uv.c +2 -1
@@ -290,7 +290,8 @@ static int make_secure_pte(pte_t *ptep, unsigned long addr,static bool should_export_before_import (struct uv_cb_header *uvcb,struct mm_struct *mm) { -return uvcb->cmd !=UVC_CMD_UNPIN_PAGE_SHARED && +return !test_bit_inv (BIT_UV_FEAT_MISC , &uv_info.uv_feature_indications) && + uvcb->cmd !=UVC_CMD_UNPIN_PAGE_SHARED &&atomic_read (&mm->context.is_protected) >1 ; }
Signature · 1 line
2.31.1
Janosch Frank
maintainerRe: [PATCH v2 13/13] KVM: s390: pv: add support for UV feature bits
Quoted 2 lines · Claudio Imbrenda
Add support for Ultravisor feature bits, and take advantage of the functionality advertised to speed up the lazy destroy mechanism.
UV feature bit support is already merged please fix the description and subject.
Quoted 21 lines
Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/kernel/uv.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/s390/kernel/uv.c +2
-1@@ -290,7 +290,8 @@ static int make_secure_pte(pte_t *ptep, unsigned long addr,static bool should_export_before_import (struct uv_cb_header *uvcb,struct mm_struct *mm) { -return uvcb->cmd !=UVC_CMD_UNPIN_PAGE_SHARED && +return !test_bit_inv (BIT_UV_FEAT_MISC , &uv_info.uv_feature_indications) && + uvcb->cmd !=UVC_CMD_UNPIN_PAGE_SHARED &&atomic_read (&mm->context.is_protected) >1 ; }
Claudio Imbrenda
maintainerRe: [PATCH v2 13/13] KVM: s390: pv: add support for UV feature bits
Quoted 6 lines · On Thu, 29 Jul 2021 11:52:58 +0200 Janosch Frank <[email protected]> wrote:
Quoted 2 lines · Claudio Imbrenda
Add support for Ultravisor feature bits, and take advantage of the functionality advertised to speed up the lazy destroy mechanism.
UV feature bit support is already merged please fix the description and subject.
oops, will be fixed
Quoted 23 lines
Quoted 22 lines
Signed-off-by Claudio Imbrenda <[email protected]>--- arch/s390/kernel/uv.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/s390/kernel/uv.c +1
-1@@ -290,7 +290,8 @@ static int make_secure_pte(pte_t *ptep, unsigned long addr, static bool should_export_before_import(struct uv_cb_header *uvcb, struct mm_struct *mm) { - return uvcb->cmd != UVC_CMD_UNPIN_PAGE_SHARED && + return !test_bit_inv(BIT_UV_FEAT_MISC,&uv_info.uv_feature_indications) &&
arch/s390/kernel/uv.c +1
-0+ uvcb->cmd !=UVC_CMD_UNPIN_PAGE_SHARED &&atomic_read (&mm->context.is_protected) >1 ; }