s390/mm: shared zeropage + KVM fixes
Subsystems (6)
v3 cover s390/mm: shared zeropage + KVM fixes
This series fixes one issue with uffd + shared zeropages on s390x and fixes that "ordinary" KVM guests can make use of shared zeropages again.
userfaultfd could currently end up mapping shared zeropages into processes that forbid shared zeropages. This only apples to s390x, relevant for handling PV guests and guests that use storage kets correctly. Fix it by placing a zeroed folio instead of the shared zeropage during UFFDIO_ZEROPAGE instead.
I stumbled over this issue while looking into a customer scenario that is using:
(1) Memory ballooning for dynamic resizing. Start a VM with, say, 100 GiB and inflate the balloon during boot to 60 GiB. The VM has ~40 GiB available and additional memory can be "fake hotplugged" to the VM later on demand by deflating the balloon. Actual memory overcommit is not desired, so physical memory would only be moved between VMs.
(2) Live migration of VMs between sites to evacuate servers in case of emergency.
Without the shared zeropage, during (2), the VM would suddenly consume 100 GiB on the migration source and destination. On the migration source, where we don't excpect memory overcommit, we could easilt end up crashing the VM during migration.
Independent of that, memory handed back to the hypervisor using "free page reporting" would end up consuming actual memory after the migration on the destination, not getting freed up until reused+freed again.
While there might be ways to optimize parts of this in QEMU, we really should just support the shared zeropage again for ordinary VMs.
We only expect legcy guests to make use of storage keys, so let's handle zeropages again when enabling storage keys or when enabling PV. To not break userfaultfd like we did in the past, don't zap the shared zeropages, but instead trigger unsharing faults, just like we do for unsharing KSM pages in break_ksm().
Unsharing faults will simply replace the shared zeropage by a zeroed anonymous folio. We can already trigger the same fault path using GUP, when trying to long-term pin a shared zeropage, but also when unmerging a KSM-placed zeropages, so this is nothing new.
Patch #1 tested on 86-64 by forcing mm_forbids_zeropage() to be 1, and running the uffd selftests.
Patch #2 tested on s390x: the live migration scenario now works as expected, and kvm-unit-tests that trigger usage of skeys work well, whereby I can see detection and unsharing of shared zeropages.
Further (as broken in v2), I tested that the shared zeropage is no longer populated after skeys are used -- that mm_forbids_zeropage() works as expected:
./s390x-run s390x/skey.elf \
-no-shutdown \
-chardev socket,id=monitor,path=/var/tmp/mon,server,nowait \
-mon chardev=monitor,mode=readline
Then, in another shell:
# cat /proc/`pgrep qemu`/smaps_rollup | grep Rss Rss: 31484 kB # echo "dump-guest-memory tmp" | sudo nc -U /var/tmp/mon ... # cat /proc/`pgrep qemu`/smaps_rollup | grep Rss Rss: 160452 kB
-> Reading guest memory does not populate the shared zeropage
Doing the same with selftest.elf (no skeys)
# cat /proc/`pgrep qemu`/smaps_rollup | grep Rss Rss: 30900 kB # echo "dump-guest-memory tmp" | sudo nc -U /var/tmp/mon ... # cat /proc/`pgrep qemu`/smaps_rollup | grep Rsstmp/mon Rss: 30924 kB
-> Reading guest memory does populate the shared zeropage
Based on s390/features. Andrew agreed that both patches can go via the s390x tree.
v2 -> v3:
* "mm/userfaultfd: don't place zeropages when zeropages are disallowed" -> Fix wrong mm_forbids_zeropage check
* "s390/mm: re-enable the shared zeropage for !PV and !skeys KVM guests" -> Fix wrong mm_forbids_zeropage define
v1 -> v2:
* "mm/userfaultfd: don't place zeropages when zeropages are disallowed" -> Minor "ret" ahndling tweaks
* "s390/mm: re-enable the shared zeropage for !PV and !skeys KVM guests" -> Added Fixes: tag
David Hildenbrand (2):
mm/userfaultfd: don't place zeropages when zeropages are disallowed s390/mm: re-enable the shared zeropage for !PV and !skeys KVM guests
arch/s390/include/asm/gmap.h | 2 +- arch/s390/include/asm/mmu.h | 5 + arch/s390/include/asm/mmu_context.h | 1 + arch/s390/include/asm/pgtable.h | 16 ++- arch/s390/kvm/kvm-s390.c | 4 +- arch/s390/mm/gmap.c | 163 +++++++++++++++++++++------- mm/userfaultfd.c | 34 ++++++ 7 files changed, 178 insertions(+), 47 deletions(-)
Signature · 1 line
2.44.0
PATCH v3 1/2 mm/userfaultfd: don't place zeropages when zeropages are disallowed
s390x must disable shared zeropages for processes running VMs, because the VMs could end up making use of "storage keys" or protected virtualization, which are incompatible with shared zeropages.
Yet, with userfaultfd it is possible to insert shared zeropages into such processes. Let's fallback to simply allocating a fresh zeroed anonymous folio and insert that instead.
mm_forbids_zeropage() was introduced in commit 593befa6ab74 ("mm: introduce mm_forbids_zeropage function"), briefly before userfaultfd went upstream.
Note that we don't want to fail the UFFDIO_ZEROPAGE request like we do for hugetlb, it would be rather unexpected. Further, we also cannot really indicated "not supported" to user space ahead of time: it could be that the MM disallows zeropages after userfaultfd was already registered.
--- mm/userfaultfd.c | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)
mm/userfaultfd.c +34 -0
@@ -316,6 +316,37 @@ static int mfill_atomic_pte_copy(pmd_t *dst_pmd, goto out; } +static int mfill_atomic_pte_zeroed_folio (pmd_t *dst_pmd, +struct vm_area_struct *dst_vma,unsigned long dst_addr) +{ +struct folio *folio; +int ret = -ENOMEM ; + + folio =vma_alloc_zeroed_movable_folio (dst_vma, dst_addr); +if (!folio) +return ret; + +if (mem_cgroup_charge (folio, dst_vma->vm_mm,GFP_KERNEL )) + goto out_put; + +/* +* The memory barrier inside __folio_mark_uptodate makes sure that +* zeroing out the folio become visible before mapping the page +* using set_pte_at(). See do_anonymous_page(). +*/ +__folio_mark_uptodate (folio); + + ret =mfill_atomic_install_pte (dst_pmd, dst_vma, dst_addr, + &folio->page, true,0 ); +if (ret) + goto out_put; + +return 0 ; +out_put: +folio_put (folio); +return ret; +} +static int mfill_atomic_pte_zeropage (pmd_t *dst_pmd,struct vm_area_struct *dst_vma,unsigned long dst_addr)
@@ -324,6 +355,9 @@ static int mfill_atomic_pte_zeropage(pmd_t *dst_pmd,spinlock_t *ptl;int ret; +if (mm_forbids_zeropage (dst_vma->vm_mm)) +return mfill_atomic_pte_zeroed_folio (dst_pmd, dst_vma, dst_addr); + _dst_pte =pte_mkspecial (pfn_pte (my_zero_pfn (dst_addr), dst_vma->vm_page_prot)); ret = -EAGAIN ;
Signature · 1 line
2.44.0
PATCH v3 2/2 s390/mm: re-enable the shared zeropage for !PV and !skeys KVM guests
In reply to David Hildenbrandcommit fa41ba0d08de ("s390/mm: avoid empty zero pages for KVM guests to avoid postcopy hangs") introduced an undesired side effect when combined with memory ballooning and VM migration: memory part of the inflated memory balloon will consume memory.
Assuming we have a 100GiB VM and inflated the balloon to 40GiB. Our VM will consume ~60GiB of memory. If we now trigger a VM migration, hypervisors like QEMU will read all VM memory. As s390x does not support the shared zeropage, we'll end up allocating for all previously-inflated memory part of the memory balloon: 50 GiB. So we might easily (unexpectedly) crash the VM on the migration source.
Even worse, hypervisors like QEMU optimize for zeropage migration to not consume memory on the migration destination: when migrating a "page full of zeroes", on the migration destination they check whether the target memory is already zero (by reading the destination memory) and avoid writing to the memory to not allocate memory: however, s390x will also allocate memory here, implying that also on the migration destination, we will end up allocating all previously-inflated memory part of the memory balloon.
This is especially bad if actual memory overcommit was not desired, when memory ballooning is used for dynamic VM memory resizing, setting aside some memory during boot that can be added later on demand. Alternatives like virtio-mem that would avoid this issue are not yet available on s390x.
There could be ways to optimize some cases in user space: before reading memory in an anonymous private mapping on the migration source, check via /proc/self/pagemap if anything is already populated. Similarly check on the migration destination before reading. While that would avoid populating tables full of shared zeropages on all architectures, it's harder to get right and performant, and requires user space changes.
Further, with posctopy live migration we must place a page, so there, "avoid touching memory to avoid allocating memory" is not really possible. (Note that a previously we would have falsely inserted shared zeropages into processes using UFFDIO_ZEROPAGE where mm_forbids_zeropage() would have actually forbidden it)
PV is currently incompatible with memory ballooning, and in the common case, KVM guests don't make use of storage keys. Instead of zapping zeropages when enabling storage keys / PV, that turned out to be problematic in the past, let's do exactly the same we do with KSM pages:
trigger unsharing faults to replace the shared zeropages by proper anonymous folios.
What about added latency when enabling storage kes? Having a lot of zeropages in applicable environments (PV, legacy guests, unittests) is unexpected. Further, KSM could today already unshare the zeropages and unmerging KSM pages when enabling storage kets would unshare the KSM-placed zeropages in the same way, resulting in the same latency.
--- arch/s390/include/asm/gmap.h | 2 +- arch/s390/include/asm/mmu.h | 5 + arch/s390/include/asm/mmu_context.h | 1 + arch/s390/include/asm/pgtable.h | 16 ++- arch/s390/kvm/kvm-s390.c | 4 +- arch/s390/mm/gmap.c | 163 +++++++++++++++++++++------- 6 files changed, 144 insertions(+), 47 deletions(-)
arch/s390/include/asm/gmap.h +1 -1
@@ -146,7 +146,7 @@ int gmap_mprotect_notify(struct gmap *, unsigned long start,void gmap_sync_dirty_log_pmd (struct gmap *gmap,unsigned long dirty_bitmap[4 ],unsigned long gaddr,unsigned long vmaddr); -int gmap_mark_unmergeable (void ); +int s390_disable_cow_sharing (void );void s390_unlist_old_asce (struct gmap *gmap);int s390_replace_asce (struct gmap *gmap);void s390_uv_destroy_pfns (unsigned long count,unsigned long *pfns);
arch/s390/include/asm/mmu.h +5 -0
@@ -32,6 +32,11 @@ typedef struct { unsigned int uses_skeys:1;/* The mmu context uses CMM. */ unsigned int uses_cmm:1; +/* +* The mmu context allows COW-sharing of memory pages (KSM, zeropage). +* Note that COW-sharing during fork() is currently always allowed. +*/ + unsigned int allow_cow_sharing:1;/* The gmaps associated with this context are allowed to use huge pages. */ unsigned int allow_gmap_hpage_1m:1; } mm_context_t;
arch/s390/include/asm/mmu_context.h +1 -0
@@ -35,6 +35,7 @@ static inline int init_new_context(struct task_struct *tsk, mm->context.has_pgste =0 ; mm->context.uses_skeys =0 ; mm->context.uses_cmm =0 ; + mm->context.allow_cow_sharing =1 ; mm->context.allow_gmap_hpage_1m =0 ;#endif switch (mm->context.asce_limit) {
arch/s390/include/asm/pgtable.h +13 -3
@@ -566,10 +566,20 @@ static inline pud_t set_pud_bit(pud_t pud, pgprot_t prot) }/* -* In the case that a guest uses storage keys -* faults should no longer be backed by zero pages +* As soon as the guest uses storage keys or enables PV, we deduplicate all +* mapped shared zeropages and prevent new shared zeropages from getting +* mapped. */ -#define mm_forbids_zeropage mm_has_pgste +#define mm_forbids_zeropage mm_forbids_zeropage +static inline int mm_forbids_zeropage (struct mm_struct *mm) +{ +#ifdef CONFIG_PGSTE +if (!mm->context.allow_cow_sharing) +return 1 ; +#endif +return 0 ; +} +static inline int mm_uses_skeys (struct mm_struct *mm) {#ifdef CONFIG_PGSTE
arch/s390/kvm/kvm-s390.c +1 -3
@@ -2631,9 +2631,7 @@ static int kvm_s390_handle_pv(struct kvm *kvm, struct kvm_pv_cmd *cmd)if (r)break ; -mmap_write_lock (current->mm); - r =gmap_mark_unmergeable (); -mmap_write_unlock (current->mm); + r =s390_disable_cow_sharing ();if (r)break ;
arch/s390/mm/gmap.c +123 -40
193 lines not on this page · raw message
Signature · 1 line
2.44.0
Alexander Gordeev
maintainerRe: [PATCH v3 2/2] s390/mm: re-enable the shared zeropage for !PV and !skeys KVM guests
On Thu, Apr 11, 2024 at 06:14:41PM +0200, David Hildenbrand wrote:
David, Christian,
Quoted 1 line
Please, correct me if I am wrong, but (to my understanding) the Tested-by for v2 does not apply for this version of the patch?
Thanks!
Re: [PATCH v3 2/2] s390/mm: re-enable the shared zeropage for !PV and !skeys KVM guests
Quoted 8 lines · Alexander Gordeev
On Thu, Apr 11, 2024 at 06:14:41PM +0200, David Hildenbrand wrote:
David, Christian,
Quoted 1 line
Please, correct me if I am wrong, but (to my understanding) the Tested-by for v2 does not apply for this version of the patch?
I thought I'd removed it -- you're absolutely, this should be dropped. Hopefully Christian has time to retest.
Thanks!
Signature · 3 lines
Cheers, David / dhildenb
Christian Borntraeger
maintainerRe: [PATCH v3 2/2] s390/mm: re-enable the shared zeropage for !PV and !skeys KVM guests
Am 11.04.24 um 23:09 schrieb David Hildenbrand:
Quoted 11 lines · Alexander Gordeev
Quoted 8 lines · David Hildenbrand
David, Christian,
Quoted 1 line
Please, correct me if I am wrong, but (to my understanding) the Tested-by for v2 does not apply for this version of the patch?
I thought I'd removed it -- you're absolutely, this should be dropped. Hopefully Christian has time to retest.
So I can confirm that this patch does continue fix the qemu memory consumption for a guest doing managedsave/start.
A quick check of other aspects seems to be ok. We will have more coverage on the base functionality as soon as it hits next(via Andrew) as our daily CI will pick this up for lots of KVM tests.
Alexander Gordeev
maintainerRe: [PATCH v3 2/2] s390/mm: re-enable the shared zeropage for !PV and !skeys KVM guests
On Mon, Apr 15, 2024 at 01:49:15PM +0200, Christian Borntraeger wrote:
Hi Christian,
Quoted 9 lines
Quoted 7 lines · Alexander Gordeev
Quoted 5 lines · David Hildenbrand
David, Christian,
Quoted 1 line
Please, correct me if I am wrong, but (to my understanding) the Tested-by for v2 does not apply for this version of the patch?
I thought I'd removed it -- you're absolutely, this should be dropped. Hopefully Christian has time to retest.
So I can confirm that this patch does continue fix the qemu memory consumption for a guest doing managedsave/start.
I will re-add your Tested-by.
Quoted 1 line
A quick check of other aspects seems to be ok. We will have more coverage on the base functionality as soon as it hits next(via Andrew) as our daily CI will pick this up for lots of KVM tests.
As per the cover letter I will pull it via s390 tree:
"Based on s390/features. Andrew agreed that both patches can go via the s390x tree."
Thanks!
Alexander Gordeev
maintainerRe: [PATCH v3 2/2] s390/mm: re-enable the shared zeropage for !PV and !skeys KVM guests
In reply to David HildenbrandOn Thu, Apr 11, 2024 at 06:14:41PM +0200, David Hildenbrand wrote:
David, could you please clarify the below questions?
Quoted 12 lines
arch/s390/mm/gmap.c +12
-0+static int __s390_unshare_zeropages (struct mm_struct *mm) +{ +struct vm_area_struct *vma; +VMA_ITERATOR (vmi, mm,0 ); +unsigned long addr; +int rc; + +for_each_vma (vmi, vma) { +/* +* We could only look at COW mappings, but it's more future +* proof to catch unexpected zeropages in other mappings and +* fail.
arch/s390/mm/gmap.c int s390_enable_sie(void) · line 2637
Quoted 10 lines
arch/s390/mm/gmap.c +10
-0+*/ +if ((vma->vm_flags &VM_PFNMAP ) ||is_vm_hugetlb_page (vma)) +continue ; + addr = vma->vm_start; + +retry: + rc =walk_page_range_vma (vma, addr, vma->vm_end, + &find_zeropage_ops, &addr); +if (rc <=0 ) +continue ;
So in case an error is returned for the last vma, __s390_unshare_zeropage() finishes with that error. By contrast, the error for a non-last vma would be ignored?
arch/s390/mm/gmap.c int s390_enable_sie(void) · line 2644
Quoted 7 lines
arch/s390/mm/gmap.c +7
-0+ +/* addr was updated by find_zeropage_pte_entry() */ + rc =handle_mm_fault (vma, addr, +FAULT_FLAG_UNSHARE |FAULT_FLAG_REMOTE , +NULL ); +if (rc &VM_FAULT_OOM ) +return -ENOMEM ;
Heiko pointed out that rc type is inconsistent vs vm_fault_t returned by handle_mm_fault(). While fixing it up, I've got concerned whether is it fine to continue in case any other error is met (including possible future VM_FAULT_xxxx)?
Quoted 6 lines
arch/s390/mm/gmap.c +6
-0+/* +* See break_ksm(): even after handle_mm_fault() returned 0, we +* must start the lookup from the current address, because +* handle_mm_fault() may back out if there's any difficulty. +* +* VM_FAULT_SIGBUS and VM_FAULT_SIGSEGV are unexpected but
arch/s390/mm/gmap.c int s390_enable_sie(void) · line 2660
Quoted 10 lines
arch/s390/mm/gmap.c +10
-0+* maybe they could trigger in the future on concurrent +* truncation. In that case, the shared zeropage would be gone +* and we can simply retry and make progress. +*/ +cond_resched (); + goto retry; + } + +return rc; +}
Thanks!
Re: [PATCH v3 2/2] s390/mm: re-enable the shared zeropage for !PV and !skeys KVM guests
Quoted 3 lines · Alexander Gordeev
On Thu, Apr 11, 2024 at 06:14:41PM +0200, David Hildenbrand wrote:
David, could you please clarify the below questions?
Sure, let me take a look if we're still missing to handle some corner cases correctly.
Quoted 19 lines
Quoted 19 lines
(quoted diff) +18
-0+static int __s390_unshare_zeropages(struct mm_struct *mm) +{ + struct vm_area_struct *vma; + VMA_ITERATOR(vmi, mm, 0); + unsigned long addr; + int rc; + + for_each_vma(vmi, vma) { + /* + * We could only look at COW mappings, but it's more future + * proof to catch unexpected zeropages in other mappings and + * fail. + */ + if ((vma->vm_flags & VM_PFNMAP) || is_vm_hugetlb_page(vma)) + continue; + addr = vma->vm_start; + +retry:
Quoted 8 lines
Review
Quoted 4 lines
(quoted diff) +4
-0+ rc = walk_page_range_vma(vma, addr, vma->vm_end, + &find_zeropage_ops, &addr); + if (rc <= 0) + continue;So in case an error is returned for the last vma,
__s390_unshare_zeropage()finishes with that error. By contrast, the error for a non-last vma would be ignored?
Right, it looks a bit off. walk_page_range_vma() shouldn't fail unless find_zeropage_pte_entry() would fail -- which would also be very unexpected.
To handle it cleanly in case we would ever get a weird zeropage where we don't expect it, we should probably just exit early.
Something like the following (not compiled, addressing the comment below):
From b97cd17a3697ac402b07fe8d0033f3c10fbd6829 Mon Sep 17 00:00:00 2001
From: David Hildenbrand <[email protected]>
Date: Mon, 15 Apr 2024 20:56:20 +0200
Subject: [PATCH] fixup
--- arch/s390/mm/gmap.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
arch/s390/mm/gmap.c +8 -5
@@ -2618,7 +2618,8 @@ static int __s390_unshare_zeropages(struct mm_struct *mm)struct vm_area_struct *vma;VMA_ITERATOR (vmi, mm,0 );unsigned long addr; -int rc; +vm_fault_t rc; +int zero_page;for_each_vma (vmi, vma) {/*
@@ -2631,9 +2632,11 @@ static int __s390_unshare_zeropages(struct mm_struct *mm) addr = vma->vm_start; retry: - rc =walk_page_range_vma (vma, addr, vma->vm_end, - &find_zeropage_ops, &addr); -if (rc <=0 ) + zero_page =walk_page_range_vma (vma, addr, vma->vm_end, + &find_zeropage_ops, &addr); +if (zero_page <0 ) +return zero_page; +else if (!zero_page)continue ;/* addr was updated by find_zeropage_pte_entry() */
@@ -2656,7 +2659,7 @@ static int __s390_unshare_zeropages(struct mm_struct *mm) goto retry; } -return rc; +return 0 ; }static int __s390_disable_cow_sharing (struct mm_struct *mm)
Signature · 1 line
2.44.0
Quoted 10 lines
Review
Quoted 8 lines
(quoted diff) +7
-0+ + /* addr was updated by find_zeropage_pte_entry() */ + rc = handle_mm_fault(vma, addr, + FAULT_FLAG_UNSHARE | FAULT_FLAG_REMOTE, + NULL); + if (rc & VM_FAULT_OOM) + return -ENOMEM;Heiko pointed out that rc type is inconsistent vs
vm_fault_treturned by
Right, let's use another variable for that.
Quoted 3 lines
handle_mm_fault(). While fixing it up, I've got concerned whether is it fine to continue in case any other error is met (including possible futureVM_FAULT_xxxx)?
Such future changes would similarly break break_ksm(). Staring at it, I do wonder if break_ksm() should be handling VM_FAULT_HWPOISON ... very likely we should handle it and fail -- we might get an MC while copying from the source page.
VM_FAULT_HWPOISON on the shared zeropage would imply a lot of trouble, so I'm not concerned about that for the case here, but handling it in the future would be cleaner.
Note that we always retry the lookup, so we won't just skip a zeropage on unexpected errors.
We could piggy-back on vm_fault_to_errno(). We could use vm_fault_to_errno(rc, FOLL_HWPOISON), and only continue (retry) if the rc is 0 or -EFAULT, otherwise fail with the returned error.
But I'd do that as a follow up, and also use it in break_ksm() in the same fashion.
Signature · 3 lines
Cheers, David / dhildenb
Alexander Gordeev
maintainerRe: [PATCH v3 2/2] s390/mm: re-enable the shared zeropage for !PV and !skeys KVM guests
Quoted 20 lines · David Hildenbrand
Quoted 9 lines
Review
Quoted 5 lines
(quoted diff) +5
-0+retry: + rc = walk_page_range_vma(vma, addr, vma->vm_end, + &find_zeropage_ops, &addr); + if (rc <= 0) + continue;So in case an error is returned for the last vma,
__s390_unshare_zeropage()finishes with that error. By contrast, the error for a non-last vma would be ignored?Right, it looks a bit off.
walk_page_range_vma()shouldn't fail unlessfind_zeropage_pte_entry()would fail -- which would also be very unexpected.To handle it cleanly in case we would ever get a weird zeropage where we don't expect it, we should probably just exit early.
Something like the following (not compiled, addressing the comment below):
arch/s390/mm/gmap.c static int __s390_unshare_zeropages(struct mm_struct *mm) · line 2622
Quoted 6 lines
arch/s390/mm/gmap.c +2
-1@@ -2618,7 +2618,8 @@ static int __s390_unshare_zeropages(struct mm_struct *mm)struct vm_area_struct *vma;VMA_ITERATOR (vmi, mm,0 );unsigned long addr; -int rc; +vm_fault_t rc; +int zero_page;
I would use "fault" for mm faults (just like everywhere else handle_mm_fault() is called) and leave rc as is:
vm_fault_t fault; int rc;
Quoted 22 lines
arch/s390/mm/gmap.c +6
-4for_each_vma (vmi, vma) {/* @@ -2631,9 +2632,11 @@ static int __s390_unshare_zeropages(struct mm_struct *mm) addr = vma->vm_start; retry: - rc =walk_page_range_vma (vma, addr, vma->vm_end, - &find_zeropage_ops, &addr); -if (rc <=0 ) + zero_page =walk_page_range_vma (vma, addr, vma->vm_end, + &find_zeropage_ops, &addr); +if (zero_page <0 ) +return zero_page; +else if (!zero_page)continue ;/* addr was updated by find_zeropage_pte_entry() */ @@ -2656,7 +2659,7 @@ static int __s390_unshare_zeropages(struct mm_struct *mm) goto retry; } -return rc; +return 0 ; }static int __s390_disable_cow_sharing (struct mm_struct *mm)
...
Quoted 23 lines
Quoted 8 lines
Review
Quoted 6 lines
(quoted diff) +6
-0+ /* addr was updated by find_zeropage_pte_entry() */ + rc = handle_mm_fault(vma, addr, + FAULT_FLAG_UNSHARE | FAULT_FLAG_REMOTE, + NULL); + if (rc & VM_FAULT_OOM) + return -ENOMEM;Heiko pointed out that rc type is inconsistent vs
vm_fault_treturned byRight, let's use another variable for that.
Quoted 3 lines
handle_mm_fault(). While fixing it up, I've got concerned whether is it fine to continue in case any other error is met (including possible futureVM_FAULT_xxxx)?Such future changes would similarly break
break_ksm(). Staring at it, I do wonder ifbreak_ksm()should be handlingVM_FAULT_HWPOISON... very likely we should handle it and fail -- we might get an MC while copying from the source page.
VM_FAULT_HWPOISONon the shared zeropage would imply a lot of trouble, so I'm not concerned about that for the case here, but handling it in the future would be cleaner.
Quoted 8 lines
Note that we always retry the lookup, so we won't just skip a zeropage on unexpected errors.
We could piggy-back on
vm_fault_to_errno(). We could usevm_fault_to_errno(rc, FOLL_HWPOISON), and only continue (retry) if the rc is 0 or-EFAULT, otherwise fail with the returned error.But I'd do that as a follow up, and also use it in
break_ksm()in the same fashion.
@Christian, do you agree with this suggestion?
Thanks!
Re: [PATCH v3 2/2] s390/mm: re-enable the shared zeropage for !PV and !skeys KVM guests
Quoted 31 lines · Alexander Gordeev
Quoted 20 lines · David Hildenbrand
Quoted 9 lines
Review
Quoted 5 lines
(quoted diff) +5
-0+retry: + rc = walk_page_range_vma(vma, addr, vma->vm_end, + &find_zeropage_ops, &addr); + if (rc <= 0) + continue;So in case an error is returned for the last vma,
__s390_unshare_zeropage()finishes with that error. By contrast, the error for a non-last vma would be ignored?Right, it looks a bit off.
walk_page_range_vma()shouldn't fail unlessfind_zeropage_pte_entry()would fail -- which would also be very unexpected.To handle it cleanly in case we would ever get a weird zeropage where we don't expect it, we should probably just exit early.
Something like the following (not compiled, addressing the comment below):
arch/s390/mm/gmap.c static int __s390_unshare_zeropages(struct mm_struct *mm) · line 2622
Quoted 6 lines
arch/s390/mm/gmap.c +2
-1@@ -2618,7 +2618,8 @@ static int __s390_unshare_zeropages(struct mm_struct *mm)struct vm_area_struct *vma;VMA_ITERATOR (vmi, mm,0 );unsigned long addr; -int rc; +vm_fault_t rc; +int zero_page;I would use "fault" for mm faults (just like everywhere else
handle_mm_fault()is called) and leave rc as is:
Quoted 2 lines
vm_fault_t fault; int rc;
Sure, let me know once discussion here stopped whether you want a v4 or can fix that up.
Signature · 3 lines
Cheers, David / dhildenb
Christian Borntraeger
maintainerRe: [PATCH v3 2/2] s390/mm: re-enable the shared zeropage for !PV and !skeys KVM guests
In reply to Alexander GordeevAm 16.04.24 um 08:37 schrieb Alexander Gordeev:
Quoted 7 lines
Quoted 5 lines
We could piggy-back on
vm_fault_to_errno(). We could usevm_fault_to_errno(rc, FOLL_HWPOISON), and only continue (retry) if the rc is 0 or-EFAULT, otherwise fail with the returned error.But I'd do that as a follow up, and also use it in
break_ksm()in the same fashion.@Christian, do you agree with this suggestion?
I would need to look into that more closely to give a proper answer. In general I am ok with this but I prefer to have more eyes on that.
From what I can tell we should cover all the normal cases with our CI as soon as it hits next. But maybe we should try to create/change a selftest to trigger these error cases?
Re: [PATCH v3 2/2] s390/mm: re-enable the shared zeropage for !PV and !skeys KVM guests
Quoted 8 lines · Christian Borntraeger
Am 16.04.24 um 08:37 schrieb Alexander Gordeev:
Quoted 4 lines
Quoted 4 lines
We could piggy-back on
vm_fault_to_errno(). We could usevm_fault_to_errno(rc, FOLL_HWPOISON), and only continue (retry) if the rc is 0 or-EFAULT, otherwise fail with the returned error.
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
But I'd do that as a follow up, and also use it in
break_ksm()in the same fashion.@Christian, do you agree with this suggestion?
I would need to look into that more closely to give a proper answer. In general I am ok with this but I prefer to have more eyes on that.
From what I can tell we should cover all the normal cases with our CI as soon as it hits
next. But maybe we should try to create/change a selftest to trigger these error cases?
If we find a shared zeropage we expect the next unsharing fault to succeed except:
(1) OOM, in which case we translate to -ENOMEM.
(2) Some obscure race with MADV_DONTNEED paired with concurrent truncate(), in which case we get an error, but if we look again, we will find the shared zeropage no longer mapped. (this is what break_ksm() describes)
(3) MCE while copying the page, which doesn't quite apply here.
For the time being, we only get shared zeropages in (a) anon mappings (b) MAP_PRIVATE shmem mappings via UFFDIO_ZEROPAGE. So (2) is hard or even impossible to trigger. (1) is hard to test as well, and (3) ...
No easy way to extend selftests that I can see.
If we repeatedly find a shared zeropage in a COW mapping and get an error from the unsharing fault, something else would be deeply flawed. So I'm not really worried about that, but I agree that having a more centralized check will make sense.
Signature · 3 lines
Cheers, David / dhildenb
Christian Borntraeger
maintainerRe: [PATCH v3 2/2] s390/mm: re-enable the shared zeropage for !PV and !skeys KVM guests
Am 16.04.24 um 15:41 schrieb David Hildenbrand:
Quoted 22 lines · Christian Borntraeger
Quoted 8 lines
Am 16.04.24 um 08:37 schrieb Alexander Gordeev:
Quoted 4 lines
Quoted 4 lines
We could piggy-back on
vm_fault_to_errno(). We could usevm_fault_to_errno(rc, FOLL_HWPOISON), and only continue (retry) if the rc is 0 or-EFAULT, otherwise fail with the returned error.Quoted 8 lines
Quoted 3 lines
Quoted 1 line
But I'd do that as a follow up, and also use it in
break_ksm()in the same fashion.@Christian, do you agree with this suggestion?
I would need to look into that more closely to give a proper answer. In general I am ok with this but I prefer to have more eyes on that.
From what I can tell we should cover all the normal cases with our CI as soon as it hits next. But maybe we should try to create/change a selftest to trigger these error cases?If we find a shared zeropage we expect the next unsharing fault to succeed except:
(1) OOM, in which case we translate to
-ENOMEM.
Quoted 7 lines
(2) Some obscure race with
MADV_DONTNEEDpaired with concurrenttruncate(), in which case we get an error, but if we look again, we will find the shared zeropage no longer mapped. (this is whatbreak_ksm()describes)(3) MCE while copying the page, which doesn't quite apply here.
For the time being, we only get shared zeropages in (a) anon mappings (b)
MAP_PRIVATEshmem mappings viaUFFDIO_ZEROPAGE. So (2) is hard or even impossible to trigger. (1) is hard to test as well, and (3) ...No easy way to extend selftests that I can see.
Yes, lets just go forward.
Quoted 2 lines
If we repeatedly find a shared zeropage in a COW mapping and get an error from the unsharing fault, something else would be deeply flawed. So I'm not really worried about that, but I agree that having a more centralized check will make sense.
Andrew Morton
maintainerQuoted 5 lines · On Thu, 11 Apr 2024 18:14:39 +0200 David Hildenbrand <[email protected]> wrote:
This series fixes one issue with uffd + shared zeropages on s390x and fixes that "ordinary" KVM guests can make use of shared zeropages again.
...
Quoted 9 lines
Without the shared zeropage, during (2), the VM would suddenly consume 100 GiB on the migration source and destination. On the migration source, where we don't excpect memory overcommit, we could easilt end up crashing the VM during migration.
Independent of that, memory handed back to the hypervisor using "free page reporting" would end up consuming actual memory after the migration on the destination, not getting freed up until reused+freed again.
Is a backport desirable?
If so, the [1/2] Fixes dates back to 2015 and the [2/2] Fixes is from 2017. Is it appropriate that the patches be backported so far back, and into different kernel versions?
Quoted 17 lines · Andrew Morton
Quoted 5 lines · On Thu, 11 Apr 2024 18:14:39 +0200 David Hildenbrand <[email protected]> wrote:
This series fixes one issue with uffd + shared zeropages on s390x and fixes that "ordinary" KVM guests can make use of shared zeropages again.
...
Quoted 9 lines
Without the shared zeropage, during (2), the VM would suddenly consume 100 GiB on the migration source and destination. On the migration source, where we don't excpect memory overcommit, we could easilt end up crashing the VM during migration.
Independent of that, memory handed back to the hypervisor using "free page reporting" would end up consuming actual memory after the migration on the destination, not getting freed up until reused+freed again.
Quoted 6 lines
Is a backport desirable?
If so, the [1/2] Fixes dates back to 2015 and the [2/2] Fixes is from 2017. Is it appropriate that the patches be backported so far back, and into different kernel versions?
[2/2] won't be easy to backport to kernels without FAULT_FLAG_UNSHARE, so I wouldn't really suggest backports to kernels before that. [1/2] might be reasonable to backport, but might require some tweaking (page vs. folio).
Signature · 3 lines
Cheers, David / dhildenb
Quoted 4 lines
David Hildenbrand (2):
mm/userfaultfd: don't place zeropages when zeropages are disallowed s390/mm: re-enable the shared zeropage for !PV and !skeys KVM guests
Quoted 8 lines
arch/s390/include/asm/gmap.h | 2 +- arch/s390/include/asm/mmu.h | 5 + arch/s390/include/asm/mmu_context.h | 1 + arch/s390/include/asm/pgtable.h | 16 ++- arch/s390/kvm/kvm-s390.c | 4 +- arch/s390/mm/gmap.c | 163 +++++++++++++++++++++------- mm/userfaultfd.c | 34 ++++++ 7 files changed, 178 insertions(+), 47 deletions(-)
Applied.
Thanks, David!
Alexander Gordeev
maintainerOn Thu, Apr 11, 2024 at 06:14:39PM +0200, David Hildenbrand wrote:
Hi David,
Quoted 2 lines
Based on s390/features. Andrew agreed that both patches can go via the s390x tree.
I am going to put on a branch this series together with the selftest:
https://lore.kernel.org/r/[email protected]
I there something in s390/features your three patches depend on? Or v6.9-rc2 contains everything needed already?
Thanks!
Quoted 4 lines · Alexander Gordeev
On Thu, Apr 11, 2024 at 06:14:39PM +0200, David Hildenbrand wrote:
Hi David,
Quoted 8 lines
Quoted 2 lines
Based on s390/features. Andrew agreed that both patches can go via the s390x tree.
I am going to put on a branch this series together with the selftest:
https://lore.kernel.org/r/[email protected]I there something in s390/features your three patches depend on? Or v6.9-rc2 contains everything needed already?
v6.9-rc2 should have all we need IIRC.
Signature · 3 lines
Cheers, David / dhildenb