s390/gmap/pgtable improve handling of keyed KVM guests
Christian Borntraeger
reviewercover s390/gmap/pgtable improve handling of keyed KVM guests
These two patches try to address stalls/timeouts that we have seen when switching many guests to enable storage keys.
Christian Borntraeger (2):
s390/gmap: voluntarily schedule during key setting s390/pgtable: use non-quiescing sske for KVM switch to keyed
arch/s390/mm/gmap.c | 14 ++++++++++++++ arch/s390/mm/pgtable.c | 2 +- 2 files changed, 15 insertions(+), 1 deletion(-)
Signature · 1 line
2.35.1
Christian Borntraeger
reviewerPATCH 1/2 s390/gmap: voluntarily schedule during key setting
With large and many guest with storage keys it is possible to create large latencies or stalls during initial key setting:
rcu: INFO: rcu_sched self-detected stall on CPU
rcu: 18-....: (2099 ticks this GP) idle=54e/1/0x4000000000000002 softirq=35598716/35598716 fqs=998
(t=2100 jiffies g=155867385 q=20879)
Task dump for CPU 18:
CPU 1/KVM R running task 0 1030947 256019 0x06000004
Call Trace:sched_show_taskrcu_dump_cpu_stacksrcu_sched_clock_irqupdate_process_timestick_sched_handletick_sched_timer__hrtimer_run_queueshrtimer_interruptdo_IRQext_int_handlerptep_zap_key
The mmap lock is held during the page walking but since this is a semaphore scheduling is still possible. Same for the kvm srcu. To minimize overhead do this on every segment table entry or large page.
--- arch/s390/mm/gmap.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
arch/s390/mm/gmap.c +14 -0
@@ -2608,6 +2608,18 @@ static int __s390_enable_skey_pte(pte_t *pte, unsigned long addr,return 0 ; } +/* +* Give a chance to schedule after setting a key to 256 pages. +* We only hold the mm lock, which is a rwsem and the kvm srcu. +* Both can sleep. +*/ +static int __s390_enable_skey_pmd (pmd_t *pmd,unsigned long addr, +unsigned long next,struct mm_walk *walk) +{ +cond_resched (); +return 0 ; +} +static int __s390_enable_skey_hugetlb (pte_t *pte,unsigned long addr,unsigned long hmask,unsigned long next,struct mm_walk *walk)
@@ -2630,12 +2642,14 @@ static int __s390_enable_skey_hugetlb(pte_t *pte, unsigned long addr, end = start +HPAGE_SIZE -1 ;__storage_key_init_range (start, end);set_bit (PG_arch_1, &page->flags); +cond_resched ();return 0 ; }static const struct mm_walk_ops enable_skey_walk_ops = { .hugetlb_entry = __s390_enable_skey_hugetlb, .pte_entry = __s390_enable_skey_pte, + .pmd_entry = __s390_enable_skey_pmd, };int s390_enable_skey (void )
Signature · 1 line
2.35.1
Re: [PATCH 1/2] s390/gmap: voluntarily schedule during key setting
Quoted 21 lines · On Mon, 30 May 2022 11:27:05 +0200 Christian Borntraeger…
With large and many guest with storage keys it is possible to create large latencies or stalls during initial key setting:
rcu: INFO: rcu_sched self-detected stall on CPU rcu: 18-....: (2099 ticks this GP) idle=54e/1/0x4000000000000002 softirq=35598716/35598716 fqs=998 (t=2100 jiffies g=155867385 q=20879) Task dump for CPU 18: CPU 1/KVM R running task 0 1030947 256019 0x06000004 Call Trace:
sched_show_taskrcu_dump_cpu_stacksrcu_sched_clock_irqupdate_process_timestick_sched_handletick_sched_timer__hrtimer_run_queueshrtimer_interruptdo_IRQext_int_handlerptep_zap_key
Quoted 5 lines
The mmap lock is held during the page walking but since this is a semaphore scheduling is still possible. Same for the kvm srcu. To minimize overhead do this on every segment table entry or large page.
Signed-off-by Christian Borntraeger <[email protected]>
Quoted 42 lines
--- arch/s390/mm/gmap.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)arch/s390/mm/gmap.c +14
-0@@ -2608,6 +2608,18 @@ static int __s390_enable_skey_pte(pte_t *pte, unsigned long addr,return 0 ; } +/* +* Give a chance to schedule after setting a key to 256 pages. +* We only hold the mm lock, which is a rwsem and the kvm srcu. +* Both can sleep. +*/ +static int __s390_enable_skey_pmd (pmd_t *pmd,unsigned long addr, +unsigned long next,struct mm_walk *walk) +{ +cond_resched (); +return 0 ; +} +static int __s390_enable_skey_hugetlb (pte_t *pte,unsigned long addr,unsigned long hmask,unsigned long next,struct mm_walk *walk)@@ -2630,12 +2642,14 @@ static int __s390_enable_skey_hugetlb(pte_t *pte, unsigned long addr, end = start +HPAGE_SIZE -1 ;__storage_key_init_range (start, end);set_bit (PG_arch_1, &page->flags); +cond_resched ();return 0 ; }static const struct mm_walk_ops enable_skey_walk_ops = { .hugetlb_entry = __s390_enable_skey_hugetlb, .pte_entry = __s390_enable_skey_pte, + .pmd_entry = __s390_enable_skey_pmd, };int s390_enable_skey (void )
Re: [PATCH 1/2] s390/gmap: voluntarily schedule during key setting
In reply to Christian BorntraegerQuoted 21 lines · Christian Borntraeger
With large and many guest with storage keys it is possible to create large latencies or stalls during initial key setting:
rcu: INFO: rcu_sched self-detected stall on CPU rcu: 18-....: (2099 ticks this GP) idle=54e/1/0x4000000000000002 softirq=35598716/35598716 fqs=998 (t=2100 jiffies g=155867385 q=20879) Task dump for CPU 18: CPU 1/KVM R running task 0 1030947 256019 0x06000004 Call Trace:
sched_show_taskrcu_dump_cpu_stacksrcu_sched_clock_irqupdate_process_timestick_sched_handletick_sched_timer__hrtimer_run_queueshrtimer_interruptdo_IRQext_int_handlerptep_zap_key
Quoted 5 lines
The mmap lock is held during the page walking but since this is a semaphore scheduling is still possible. Same for the kvm srcu. To minimize overhead do this on every segment table entry or large page.
Signed-off-by Christian Borntraeger <[email protected]>
Quoted 45 lines
--- arch/s390/mm/gmap.c | 14 ++++++++++++++ 1 file changed, 14 insertions(+)arch/s390/mm/gmap.c +14
-0@@ -2608,6 +2608,18 @@ static int __s390_enable_skey_pte(pte_t *pte, unsigned long addr,return 0 ; } +/* +* Give a chance to schedule after setting a key to 256 pages. +* We only hold the mm lock, which is a rwsem and the kvm srcu. +* Both can sleep. +*/ +static int __s390_enable_skey_pmd (pmd_t *pmd,unsigned long addr, +unsigned long next,struct mm_walk *walk) +{ +cond_resched (); +return 0 ; +} +static int __s390_enable_skey_hugetlb (pte_t *pte,unsigned long addr,unsigned long hmask,unsigned long next,struct mm_walk *walk)@@ -2630,12 +2642,14 @@ static int __s390_enable_skey_hugetlb(pte_t *pte, unsigned long addr, end = start +HPAGE_SIZE -1 ;__storage_key_init_range (start, end);set_bit (PG_arch_1, &page->flags); +cond_resched ();return 0 ; }static const struct mm_walk_ops enable_skey_walk_ops = { .hugetlb_entry = __s390_enable_skey_hugetlb, .pte_entry = __s390_enable_skey_pte, + .pmd_entry = __s390_enable_skey_pmd, };int s390_enable_skey (void )Signature · 1 line
2.35.1
Christian Borntraeger
reviewerPATCH 2/2 s390/pgtable: use non-quiescing sske for KVM switch to keyed
In reply to Christian BorntraegerThe switch to a keyed guest does not require a classic sske as the other guest CPUs are not accessing the key before the switch is complete. By using the NQ SSKE things are faster especially with multiple guests.
--- arch/s390/mm/pgtable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/s390/mm/pgtable.c +1 -1
@@ -748,7 +748,7 @@ void ptep_zap_key(struct mm_struct *mm, unsigned long addr, pte_t *ptep)pgste_val (pgste) |=PGSTE_GR_BIT |PGSTE_GC_BIT ; ptev =pte_val (*ptep);if (!(ptev & _PAGE_INVALID) && (ptev & _PAGE_WRITE)) -page_set_storage_key (ptev &PAGE_MASK ,PAGE_DEFAULT_KEY ,1 ); +page_set_storage_key (ptev &PAGE_MASK ,PAGE_DEFAULT_KEY ,0 );pgste_set_unlock (ptep, pgste);preempt_enable (); }
Signature · 1 line
2.35.1
Re: [PATCH 2/2] s390/pgtable: use non-quiescing sske for KVM switch to keyed
Quoted 6 lines · On Mon, 30 May 2022 11:27:06 +0200 Christian Borntraeger…
The switch to a keyed guest does not require a classic sske as the other guest CPUs are not accessing the key before the switch is complete. By using the NQ SSKE things are faster especially with multiple guests.
Signed-off-by Christian Borntraeger <[email protected]>Suggested-by Janis Schoetterl-Glausch <[email protected]>
Quoted 17 lines
--- arch/s390/mm/pgtable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/s390/mm/pgtable.c +1
-1@@ -748,7 +748,7 @@ void ptep_zap_key(struct mm_struct *mm, unsigned long addr, pte_t *ptep)pgste_val (pgste) |=PGSTE_GR_BIT |PGSTE_GC_BIT ; ptev =pte_val (*ptep);if (!(ptev & _PAGE_INVALID) && (ptev & _PAGE_WRITE)) -page_set_storage_key (ptev &PAGE_MASK ,PAGE_DEFAULT_KEY ,1 ); +page_set_storage_key (ptev &PAGE_MASK ,PAGE_DEFAULT_KEY ,0 );pgste_set_unlock (ptep, pgste);preempt_enable (); }
Christian Borntraeger
reviewerAm 30.05.22 um 11:27 schrieb Christian Borntraeger:
Quoted 10 lines
These two patches try to address stalls/timeouts that we have seen when switching many guests to enable storage keys.
Christian Borntraeger (2):
s390/gmap: voluntarily schedule during key setting s390/pgtable: use non-quiescing sske for KVM switch to keyedarch/s390/mm/gmap.c | 14 ++++++++++++++ arch/s390/mm/pgtable.c | 2 +- 2 files changed, 15 insertions(+), 1 deletion(-)
queued.