KVM: s390: make use of the GIB
Subsystems (5)
v5 cover KVM: s390: make use of the GIB
The Guest Information Area (GIB) and its mechanics is part of the AIV facility. It provides hardware support to process Adapter Interruptions (AI) for pagable storage mode guests.
Whenever a guest cannot process an AI because none of its vcpus is running in SIE context, a GIB alert interruption is sent to and handled by the host program to dispatch a vcpu of the respective guest.
Eventually, the AI is handled by the guest.
v4->v5:
Between these two versions some significant change was introduced:
a) During IPM look-up, the IAM gets atomically restored if the IPM is clean. (The main strategy is: As soon no airqs are pending anymore, get prepared for new ones arriving).
b) Kick a set of vcpus in WAIT state, that will be able to handle all ISCs in IPM if possible. The main loop that processes the GIB alert list is unchanged.
patches 1/15 to 6/15:
prepare the existing Interruption and GISA code for the introduction of the GIB code.
patches 7/15 to 9/15:
kept from v4.
patch 10/15:
restores the IAM in get_ipm() when clean on request with additional parameter irq_flags
patch 12/15:
modifies kvm_s390_deliver_pending_interrupts() such that the IAM is not restored during the deliverable_irqs() test as we are about to enter the SIE. Restoring the IAM would trigger the alert mechanism unnecessarily on SIE entry.
patch 13/15:
process_gib_alert_list() now triggers a minimal set of idle vcpus capable the handle all pending ISCs.
patch 14/15:
the wiring is logical similar to v4 but uses different routines
patch 15/15:
kept from v4.
v3->v4:
The central change of v4 is that the metric "vcpus_in_sie" has been dropped. A least agressive IAM restore strategy is being followed now. The consequence is that potentially more GAL interruptions are to be handled by the host program. The assuption as made that as soon the first vcpu is trying to go into wait state, this could be the last vcpu being open for I/O interruptions. Thus, to not loose the initiative to deliver I/O interruptions as soon as possible, the IAM is restored. Adding better heuristics can be done in future.
I expect this to be the last version of the series.
patch 01/12: Kernel msg now printed when FLIC fails
to register during arch inititialization.
patch 03/12: Commit message slightly changed with the
hint that gib initialization is called by last patch of this serias.
patch 09/12: IAM now restored during kvm_s390_handle_wait() patch 10/12: gib destroy case now handled upon unsuccessful
arch inititialization.
All other patches are unchanged.
v2->v3:
patch 01/12: Fixes a resource dealocation issue upon
unsuccessful exit from kvm_arch_init(). It is relevant for patch 11/12 as well.
patch 08/12: The function process_gib_alert_list() has
been rewritten to reduce the number of GAL interruptions on the host for many guest scenarios.
patch 10/12: The IAM is now never cleared before SIE entry.
It will be cleared by millicode as soon the first guest interruption is made pending and not directly processed in SIE. The IAM gets restored as soon a vm is idle, i.e. no vcpu of that guest is in SIE context anymore.
patch 11/12: now integrates with new patch 01/12 patch 12/12: This patch is just experimental and shall not
be part of the final series.
The first patch of series v2: "leave AIs in IPM of GISA during vcpu_pre_run()" has been dropped.
All other patches are unchanged.
v1->v2:
patch 01/12: New patch. Tt can go also standalone without the
rest of the GIB series presented here but is a requirement
patch 03/12: kvm_s390_gib_init() now has a return code patch 06/12: unlink_gib_alert_list() now uses cmpxchg() instead
of atomic_xchg()
patch 08/12: New patch. Foctors out __find_vcpu_for_floating_irq() patch 09/12: process_gib_alert_list() has been simplified
the GISA clear/destroy cases have been removed
patch 11/12: kvm_s390_gisa_clear/destroy() now clear the IAM
and process the alert list directly
The IAM now gets restored in vcpu_post_run() only if the GISA is not in alert list
patch 12/12: during kvm_arch_init() now the return code of
kvm_s390_gib_init() is honored.
All other patches are unchanged.
Michael Mueller (15):
KVM: s390: unregister debug feature on failing arch init
KVM: s390: coding style issue kvm_s390_gisa_init/clear()
KVM: s390: factor out nullify_gisa()
KVM: s390: use pending_irqs_no_gisa() where appropriate
KVM: s390: unify pending_irqs() and pending_irqs_no_gisa()
KVM: s390: remove prefix kvm_s390_gisa_ from static inline functions s390/cio: add function chsc_sgib()
KVM: s390: add the GIB and its related life-cyle functions
KVM: s390: add kvm reference to struct sie_page2
KVM: s390: add functions to (un)register GISC with GISA
KVM: s390: restore IAM in get_ipm() when IPM is clean
KVM: s390: do not restore IAM immediately before SIE entry
KVM: s390: add function process_gib_alert_list()
KVM: s390: add and wire function gib_alert_irq_handler()
KVM: s390: start using the GIB
Documentation/kmsg/s390/kvm | 12 ++ arch/s390/include/asm/cio.h | 1 + arch/s390/include/asm/irq.h | 1 + arch/s390/include/asm/isc.h | 1 + arch/s390/include/asm/kvm_host.h | 22 ++- arch/s390/kernel/irq.c | 1 + arch/s390/kvm/interrupt.c | 396 +++++++++++++++++++++++++++++++++++---- arch/s390/kvm/kvm-s390.c | 25 ++- arch/s390/kvm/kvm-s390.h | 2 + drivers/s390/cio/chsc.c | 37 ++++ drivers/s390/cio/chsc.h | 1 + 11 files changed, 457 insertions(+), 42 deletions(-) create mode 100644 Documentation/kmsg/s390/kvm
Signature · 1 line
2.13.4
PATCH v5 1/15 KVM: s390: unregister debug feature on failing arch init
Make sure the debug feature and its allocated resources get released upon unsuccessful architecture initialization.
A related indication of the issue will be reported as kernel message.
Message-Id: <[email protected]>
--- Documentation/kmsg/s390/kvm | 12 ++++++++++++ arch/s390/kvm/kvm-s390.c | 17 ++++++++++++++--- 2 files changed, 26 insertions(+), 3 deletions(-) create mode 100644 Documentation/kmsg/s390/kvm
Documentation/kmsg/s390/kvm new file +12 -0
@@ -0,0 +1,12 @@ +/*? + * Text: "Failed to register FLIC rc=%d\n" + * Severity: Error + * Parameter: + * @1: return code of the FLIC registration call + * Description: + * The registration of the FLIC (Floating Interrupt Controller Interface) + * was not successful. + * User action: + * If this problem persists after a reload of the kvm kernel module, gather + * Linux debug data and contact your support organization. + */
arch/s390/kvm/kvm-s390.c +14 -3
@@ -417,19 +417,30 @@ static void kvm_s390_cpu_feat_init(void)int kvm_arch_init (void *opaque) { +int rc; + kvm_s390_dbf =debug_register ("kvm-trace" ,32 ,1 ,7 *sizeof (long ));if (!kvm_s390_dbf)return -ENOMEM ;if (debug_register_view (kvm_s390_dbf, &debug_sprintf_view)) { -debug_unregister (kvm_s390_dbf); -return -ENOMEM ; + rc = -ENOMEM ; + goto out_debug_unreg; }kvm_s390_cpu_feat_init ();/* Register floating interrupt controller interface. */ -return kvm_register_device_ops (&kvm_flic_ops,KVM_DEV_TYPE_FLIC ); + rc =kvm_register_device_ops (&kvm_flic_ops,KVM_DEV_TYPE_FLIC ); +if (rc) { +pr_err ("Failed to register FLIC rc=%d\n" , rc); + goto out_debug_unreg; + } +return 0 ; + +out_debug_unreg: +debug_unregister (kvm_s390_dbf); +return rc; }void kvm_arch_exit (void )
Signature · 1 line
2.13.4
Re: [PATCH v5 01/15] KVM: s390: unregister debug feature on failing arch init
Quoted 26 lines · On Wed, 19 Dec 2018 20:17:42 +0100 Michael Mueller <[email protected]> wrote:
Make sure the debug feature and its allocated resources get released upon unsuccessful architecture initialization.
A related indication of the issue will be reported as kernel message.
Signed-off-by Michael Mueller <[email protected]>Reviewed-by Cornelia Huck <[email protected]>Reviewed-by Pierre Morel <[email protected]>Reviewed-by David Hildenbrand <[email protected]>Message-Id: <[email protected]>
Signed-off-by Christian Borntraeger <[email protected]>--- Documentation/kmsg/s390/kvm | 12 ++++++++++++ arch/s390/kvm/kvm-s390.c | 17 ++++++++++++++--- 2 files changed, 26 insertions(+), 3 deletions(-) create mode 100644 Documentation/kmsg/s390/kvmDocumentation/kmsg/s390/kvm new file +2
-0@@ -0,0 +1,12 @@ +/*? + * Text: "Failed to register FLIC rc=%d\n"
Documentation/kmsg/s390/kvm line 12
Quoted 10 lines
Documentation/kmsg/s390/kvm new file +10
-0+ * Severity: Error + * Parameter: + * @1: return code of the FLIC registration call + * Description: + * The registration of the FLIC (Floating Interrupt Controller Interface) + * was not successful. + * User action: + * If this problem persists after a reload of the kvm kernel module, gather + * Linux debug data and contact your support organization. + */
Hm, it seems that the kmsg stuff crept in there again...
Re: [PATCH v5 01/15] KVM: s390: unregister debug feature on failing arch init
Quoted 33 lines · Cornelia Huck
Quoted 30 lines · On Wed, 19 Dec 2018 20:17:42 +0100 Michael Mueller <[email protected]> wrote:
Make sure the debug feature and its allocated resources get released upon unsuccessful architecture initialization.
A related indication of the issue will be reported as kernel message.
Signed-off-by Michael Mueller <[email protected]>Reviewed-by Cornelia Huck <[email protected]>Reviewed-by Pierre Morel <[email protected]>Reviewed-by David Hildenbrand <[email protected]>Message-Id: <[email protected]>
Signed-off-by Christian Borntraeger <[email protected]>--- Documentation/kmsg/s390/kvm | 12 ++++++++++++ arch/s390/kvm/kvm-s390.c | 17 ++++++++++++++--- 2 files changed, 26 insertions(+), 3 deletions(-)create mode 100644
Documentation/kmsg/s390/kvmDocumentation/kmsg/s390/kvm new file +6
-0@@ -0,0 +1,12 @@ +/*? + * Text: "Failed to register FLIC rc=%d\n" + * Severity: Error + * Parameter: + * @1: return code of the FLIC registration call + * Description:
Quoted 9 lines
Documentation/kmsg/s390/kvm line 12
Quoted 6 lines
Documentation/kmsg/s390/kvm new file +6
-0+ * The registration of the FLIC (Floating Interrupt Controller Interface) + * was not successful. + * User action: + * If this problem persists after a reload of the kvm kernel module, gather + * Linux debug data and contact your support organization. + */Hm, it seems that the kmsg stuff crept in there again...
picked from wrong branch, sorry...
Re: [PATCH v5 01/15] KVM: s390: unregister debug feature on failing arch init
Quoted 46 lines · Michael Mueller
Quoted 33 lines · Cornelia Huck
Quoted 30 lines · On Wed, 19 Dec 2018 20:17:42 +0100 Michael Mueller <[email protected]> wrote:
Make sure the debug feature and its allocated resources get released upon unsuccessful architecture initialization.
A related indication of the issue will be reported as kernel message.
Signed-off-by Michael Mueller <[email protected]>Reviewed-by Cornelia Huck <[email protected]>Reviewed-by Pierre Morel <[email protected]>Reviewed-by David Hildenbrand <[email protected]>Message-Id: <[email protected]>
Signed-off-by Christian Borntraeger <[email protected]>---
Documentation/kmsg/s390/kvm| 12 ++++++++++++ arch/s390/kvm/kvm-s390.c | 17 ++++++++++++++---2 files changed, 26 insertions(+), 3 deletions(-)create mode 100644
Documentation/kmsg/s390/kvmDocumentation/kmsg/s390/kvm new file +6
-0@@ -0,0 +1,12 @@ +/*? + * Text: "Failed to register FLIC rc=%d\n" + * Severity: Error + * Parameter: + * @1: return code of the FLIC registration call + * Description:Quoted 9 lines
Documentation/kmsg/s390/kvm line 12
Quoted 6 lines
Documentation/kmsg/s390/kvm new file +6
-0+ * The registration of the FLIC (Floating Interrupt Controller Interface) + * was not successful. + * User action: + * If this problem persists after a reload of the kvm kernel module, gather + * Linux debug data and contact your support organization. + */Hm, it seems that the kmsg stuff crept in there again...
Quoted 2 lines
picked from wrong branch, sorry...
FWIW, this patch without kmsg one is now part of kvm/next.
PATCH v5 2/15 KVM: s390: coding style issue kvm_s390_gisa_init/clear()
In reply to Michael MuellerThe change hepls to reduce line length and
icreases code readability.
--- arch/s390/kvm/interrupt.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)
arch/s390/kvm/interrupt.c +10 -10
@@ -2886,20 +2886,20 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len)void kvm_s390_gisa_clear (struct kvm *kvm) { -if (kvm->arch.gisa) { -memset (kvm->arch.gisa,0 ,sizeof (struct kvm_s390_gisa )); - kvm->arch.gisa->next_alert = (u32 )(u64 )kvm->arch.gisa; -VM_EVENT (kvm,3 ,"gisa 0x%pK cleared" , kvm->arch.gisa); - } +if (!kvm->arch.gisa) +return ; +memset (kvm->arch.gisa,0 ,sizeof (struct kvm_s390_gisa )); + kvm->arch.gisa->next_alert = (u32 )(u64 )kvm->arch.gisa; +VM_EVENT (kvm,3 ,"gisa 0x%pK cleared" , kvm->arch.gisa); }void kvm_s390_gisa_init (struct kvm *kvm) { -if (css_general_characteristics.aiv) { - kvm->arch.gisa = &kvm->arch.sie_page2->gisa; -VM_EVENT (kvm,3 ,"gisa 0x%pK initialized" , kvm->arch.gisa); -kvm_s390_gisa_clear (kvm); - } +if (!css_general_characteristics.aiv) +return ; + kvm->arch.gisa = &kvm->arch.sie_page2->gisa; +VM_EVENT (kvm,3 ,"gisa 0x%pK initialized" , kvm->arch.gisa); +kvm_s390_gisa_clear (kvm); }void kvm_s390_gisa_destroy (struct kvm *kvm)
Signature · 1 line
2.13.4
Re: [PATCH v5 02/15] KVM: s390: coding style issue kvm_s390_gisa_init/clear()
Quoted 1 line · On Wed, 19 Dec 2018 20:17:43 +0100 Michael Mueller <[email protected]> wrote:
The change hepls to reduce line length and
s/hepls/helps/
Quoted 1 line
icreases code readability.
s/icreases/increases/
Quoted 6 lines
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)
Re: [PATCH v5 02/15] KVM: s390: coding style issue kvm_s390_gisa_init/clear()
In reply to Michael MuellerQuoted 29 lines · Michael Mueller
The change hepls to reduce line length and
icreases code readability.Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)arch/s390/kvm/interrupt.c +5
-5@@ -2886,20 +2886,20 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len)void kvm_s390_gisa_clear (struct kvm *kvm) { -if (kvm->arch.gisa) { -memset (kvm->arch.gisa,0 ,sizeof (struct kvm_s390_gisa )); - kvm->arch.gisa->next_alert = (u32 )(u64 )kvm->arch.gisa; -VM_EVENT (kvm,3 ,"gisa 0x%pK cleared" , kvm->arch.gisa); - } +if (!kvm->arch.gisa) +return ; +memset (kvm->arch.gisa,0 ,sizeof (struct kvm_s390_gisa )); + kvm->arch.gisa->next_alert = (u32 )(u64 )kvm->arch.gisa; +VM_EVENT (kvm,3 ,"gisa 0x%pK cleared" , kvm->arch.gisa); }void kvm_s390_gisa_init (struct kvm *kvm)
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2901
Quoted 10 lines
arch/s390/kvm/interrupt.c +4
-5{ -if (css_general_characteristics.aiv) { - kvm->arch.gisa = &kvm->arch.sie_page2->gisa; -VM_EVENT (kvm,3 ,"gisa 0x%pK initialized" , kvm->arch.gisa); -kvm_s390_gisa_clear (kvm); - } +if (!css_general_characteristics.aiv) +return ; + kvm->arch.gisa = &kvm->arch.sie_page2->gisa; +VM_EVENT (kvm,3 ,"gisa 0x%pK initialized" , kvm->arch.gisa);
to be annoying, shouldn't the trace be done after the clear?
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2905
Quoted 5 lines
arch/s390/kvm/interrupt.c +1
-0+kvm_s390_gisa_clear (kvm); }void kvm_s390_gisa_destroy (struct kvm *kvm)
Signature · 2 lines
Pierre Morel Linux/KVM/QEMU in Böblingen - Germany
Re: [PATCH v5 02/15] KVM: s390: coding style issue kvm_s390_gisa_init/clear()
Quoted 33 lines · Pierre Morel
Quoted 32 lines · Michael Mueller
The change hepls to reduce line length and
icreases code readability.Signed-off-by Michael Mueller <[email protected]>---
arch/s390/kvm/interrupt.c| 20 ++++++++++----------1 file changed, 10 insertions(+), 10 deletions(-)arch/s390/kvm/interrupt.c +5
-8@@ -2886,20 +2886,20 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) void kvm_s390_gisa_clear(struct kvm *kvm) { - if (kvm->arch.gisa) { - memset(kvm->arch.gisa, 0, sizeof(struct kvm_s390_gisa)); - kvm->arch.gisa->next_alert = (u32)(u64)kvm->arch.gisa; - VM_EVENT(kvm, 3, "gisa 0x%pK cleared", kvm->arch.gisa); - } + if (!kvm->arch.gisa) + return; + memset(kvm->arch.gisa, 0, sizeof(struct kvm_s390_gisa)); + kvm->arch.gisa->next_alert = (u32)(u64)kvm->arch.gisa; + VM_EVENT(kvm, 3, "gisa 0x%pK cleared", kvm->arch.gisa); } void kvm_s390_gisa_init(struct kvm *kvm) { - if (css_general_characteristics.aiv) { - kvm->arch.gisa = &kvm->arch.sie_page2->gisa; - VM_EVENT(kvm, 3, "gisa 0x%pK initialized", kvm->arch.gisa);
Quoted 8 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2901
Quoted 6 lines
arch/s390/kvm/interrupt.c +4
-2- kvm_s390_gisa_clear(kvm); - } + if (!css_general_characteristics.aiv) + return; + kvm->arch.gisa = &kvm->arch.sie_page2->gisa; + VM_EVENT(kvm, 3, "gisa 0x%pK initialized", kvm->arch.gisa);to be annoying, shouldn't the trace be done after the clear?
sure, I will address that
Quoted 9 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len)
Quoted 5 lines
arch/s390/kvm/interrupt.c +1
-0+ kvm_s390_gisa_clear(kvm); } void kvm_s390_gisa_destroy(struct kvm *kvm)Reviewed-by Pierre Morel<[email protected]>
PATCH v5 3/15 KVM: s390: factor out nullify_gisa()
In reply to Michael MuellerThis function will be used by the GISA init and the GISA clear operation. Thus it gets factored out here.
--- arch/s390/kvm/interrupt.c | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
arch/s390/kvm/interrupt.c +8 -3
@@ -2884,12 +2884,17 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len)return n; } +static void nullify_gisa (struct kvm_s390_gisa *gisa) +{ +memset (gisa,0 ,sizeof (struct kvm_s390_gisa )); + gisa->next_alert = (u32 )(u64 )gisa; +} +void kvm_s390_gisa_clear (struct kvm *kvm) {if (!kvm->arch.gisa)return ; -memset (kvm->arch.gisa,0 ,sizeof (struct kvm_s390_gisa )); - kvm->arch.gisa->next_alert = (u32 )(u64 )kvm->arch.gisa; +nullify_gisa (kvm->arch.gisa);VM_EVENT (kvm,3 ,"gisa 0x%pK cleared" , kvm->arch.gisa); }
@@ -2898,8 +2903,8 @@ void kvm_s390_gisa_init(struct kvm *kvm)if (!css_general_characteristics.aiv)return ; kvm->arch.gisa = &kvm->arch.sie_page2->gisa; +nullify_gisa (kvm->arch.gisa);VM_EVENT (kvm,3 ,"gisa 0x%pK initialized" , kvm->arch.gisa); -kvm_s390_gisa_clear (kvm); }void kvm_s390_gisa_destroy (struct kvm *kvm)
Signature · 1 line
2.13.4
PATCH v5 4/15 KVM: s390: use pending_irqs_no_gisa() where appropriate
In reply to Michael MuellerInterruption types that are not represented in GISA shall use pending_irqs_no_gisa() to test pending interruptions.
--- arch/s390/kvm/interrupt.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/s390/kvm/interrupt.c +2 -2
@@ -353,7 +353,7 @@ static void set_intercept_indicators_io(struct kvm_vcpu *vcpu)static void set_intercept_indicators_ext (struct kvm_vcpu *vcpu) { -if (!(pending_irqs (vcpu) &IRQ_PEND_EXT_MASK )) +if (!(pending_irqs_no_gisa (vcpu) &IRQ_PEND_EXT_MASK ))return ;if (psw_extint_disabled (vcpu))kvm_s390_set_cpuflags (vcpu,CPUSTAT_EXT_INT );
@@ -363,7 +363,7 @@ static void set_intercept_indicators_ext(struct kvm_vcpu *vcpu)static void set_intercept_indicators_mchk (struct kvm_vcpu *vcpu) { -if (!(pending_irqs (vcpu) &IRQ_PEND_MCHK_MASK )) +if (!(pending_irqs_no_gisa (vcpu) &IRQ_PEND_MCHK_MASK ))return ;if (psw_mchk_disabled (vcpu)) vcpu->arch.sie_block->ictl |=ICTL_LPSW ;
Signature · 1 line
2.13.4
Re: [PATCH v5 04/15] KVM: s390: use pending_irqs_no_gisa() where appropriate
Quoted 8 lines · On Wed, 19 Dec 2018 20:17:45 +0100 Michael Mueller <[email protected]> wrote:
Interruption types that are not represented in GISA shall use
pending_irqs_no_gisa()to test pending interruptions.Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Re: [PATCH v5 04/15] KVM: s390: use pending_irqs_no_gisa() where appropriate
In reply to Michael MuellerQuoted 23 lines · Michael Mueller
Interruption types that are not represented in GISA shall use
pending_irqs_no_gisa()to test pending interruptions.Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/s390/kvm/interrupt.c +1
-1@@ -353,7 +353,7 @@ static void set_intercept_indicators_io(struct kvm_vcpu *vcpu)static void set_intercept_indicators_ext (struct kvm_vcpu *vcpu) { -if (!(pending_irqs (vcpu) &IRQ_PEND_EXT_MASK )) +if (!(pending_irqs_no_gisa (vcpu) &IRQ_PEND_EXT_MASK ))return ;if (psw_extint_disabled (vcpu))kvm_s390_set_cpuflags (vcpu,CPUSTAT_EXT_INT );
arch/s390/kvm/interrupt.c static void set_intercept_indicators_ext(struct kvm_vcpu *vcpu) · line 369
Quoted 8 lines
arch/s390/kvm/interrupt.c +1
-1@@ -363,7 +363,7 @@ static void set_intercept_indicators_ext(struct kvm_vcpu *vcpu)static void set_intercept_indicators_mchk (struct kvm_vcpu *vcpu) { -if (!(pending_irqs (vcpu) &IRQ_PEND_MCHK_MASK )) +if (!(pending_irqs_no_gisa (vcpu) &IRQ_PEND_MCHK_MASK ))return ;if (psw_mchk_disabled (vcpu)) vcpu->arch.sie_block->ictl |=ICTL_LPSW ;
Signature · 2 lines
Pierre Morel Linux/KVM/QEMU in Böblingen - Germany
PATCH v5 5/15 KVM: s390: unify pending_irqs() and pending_irqs_no_gisa()
In reply to Michael MuellerUse a single function with parameter irq_flags to differentiate between cases.
New irq flag:
IRQ_FLAG_LOCAL: include vcpu local interruptions pending
IRQ_FLAG_FLOATING: include vcpu floating interruptions pending
IRQ_FLAG_GISA: include GISA interruptions pending in IPM
New irq masks:
IRQ_MASK_ALL: include all types
IRQ_MASK_NO_GISA: include all types but GISA
Examples:
pending_irqs(vcpu, IRQ_MASK_ALL)
pending_irqs(vcpu, IRQ_MASK_NO_GISA)
There will be more irq flags with upcoming patches.
--- arch/s390/kvm/interrupt.c | 33 +++++++++++++++++++++------------ 1 file changed, 21 insertions(+), 12 deletions(-)
arch/s390/kvm/interrupt.c +21 -12
@@ -31,6 +31,13 @@#define PFAULT_DONE 0x0680#define VIRTIO_PARAM 0x0d00 +#define IRQ_FLAG_LOCAL 0x8000/* include local interruption pending mask */ +#define IRQ_FLAG_FLOATING 0x4000/* include float interruption pending mask */ +#define IRQ_FLAG_GISA 0x2000/* include GISA interruption pending mask */ + +#define IRQ_MASK_ALL (IRQ_FLAG_LOCAL | IRQ_FLAG_FLOATING | IRQ_FLAG_GISA) +#define IRQ_MASK_NO_GISA (IRQ_MASK_ALL & ~IRQ_FLAG_GISA) +/* handle external calls via sigp interpretation facility */ static int sca_ext_call_pending (struct kvm_vcpu *vcpu,int *src_id) {
@@ -237,16 +244,18 @@ static inline int kvm_s390_gisa_tac_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gisreturn test_and_clear_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); } -static inline unsigned long pending_irqs_no_gisa (struct kvm_vcpu *vcpu) +static inline unsigned long pending_irqs (struct kvm_vcpu *vcpu,u16 irq_flags) { -return vcpu->kvm->arch.float_int.pending_irqs | - vcpu->arch.local_int.pending_irqs; -} +unsigned long pending_irqs =0 ; -static inline unsigned long pending_irqs (struct kvm_vcpu *vcpu) -{ -return pending_irqs_no_gisa (vcpu) | -kvm_s390_gisa_get_ipm (vcpu->kvm->arch.gisa) <<IRQ_PEND_IO_ISC_7 ; +if (irq_flags &IRQ_FLAG_LOCAL ) + pending_irqs |= vcpu->arch.local_int.pending_irqs; +if (irq_flags &IRQ_FLAG_FLOATING ) + pending_irqs |= vcpu->kvm->arch.float_int.pending_irqs; +if (irq_flags &IRQ_FLAG_GISA ) + pending_irqs |=kvm_s390_gisa_get_ipm (vcpu->kvm->arch.gisa) << +IRQ_PEND_IO_ISC_7 ; +return pending_irqs; }static inline int isc_to_irq_type (unsigned long isc)
@@ -275,7 +284,7 @@ static unsigned long deliverable_irqs(struct kvm_vcpu *vcpu) {unsigned long active_mask; - active_mask =pending_irqs (vcpu); + active_mask =pending_irqs (vcpu,IRQ_MASK_ALL );if (!active_mask)return 0 ;
@@ -343,7 +352,7 @@ static void __reset_intercept_indicators(struct kvm_vcpu *vcpu)static void set_intercept_indicators_io (struct kvm_vcpu *vcpu) { -if (!(pending_irqs_no_gisa (vcpu) &IRQ_PEND_IO_MASK )) +if (!(pending_irqs (vcpu,IRQ_MASK_NO_GISA ) &IRQ_PEND_IO_MASK ))return ;else if (psw_ioint_disabled (vcpu))kvm_s390_set_cpuflags (vcpu,CPUSTAT_IO_INT );
@@ -353,7 +362,7 @@ static void set_intercept_indicators_io(struct kvm_vcpu *vcpu)static void set_intercept_indicators_ext (struct kvm_vcpu *vcpu) { -if (!(pending_irqs_no_gisa (vcpu) &IRQ_PEND_EXT_MASK )) +if (!(pending_irqs (vcpu,IRQ_MASK_NO_GISA ) &IRQ_PEND_EXT_MASK ))return ;if (psw_extint_disabled (vcpu))kvm_s390_set_cpuflags (vcpu,CPUSTAT_EXT_INT );
@@ -363,7 +372,7 @@ static void set_intercept_indicators_ext(struct kvm_vcpu *vcpu)static void set_intercept_indicators_mchk (struct kvm_vcpu *vcpu) { -if (!(pending_irqs_no_gisa (vcpu) &IRQ_PEND_MCHK_MASK )) +if (!(pending_irqs (vcpu,IRQ_MASK_NO_GISA ) &IRQ_PEND_MCHK_MASK ))return ;if (psw_mchk_disabled (vcpu)) vcpu->arch.sie_block->ictl |=ICTL_LPSW ;
Signature · 1 line
2.13.4
Re: [PATCH v5 05/15] KVM: s390: unify pending_irqs() and pending_irqs_no_gisa()
arch/s390/kvm/interrupt.c static inline int kvm_s390_gisa_tac_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gis · line 255
Quoted 10 lines · Michael Mueller
arch/s390/kvm/interrupt.c +5
-4-static inline unsigned long pending_irqs (struct kvm_vcpu *vcpu) -{ -return pending_irqs_no_gisa (vcpu) | -kvm_s390_gisa_get_ipm (vcpu->kvm->arch.gisa) <<IRQ_PEND_IO_ISC_7 ; +if (irq_flags &IRQ_FLAG_LOCAL ) + pending_irqs |= vcpu->arch.local_int.pending_irqs; +if (irq_flags &IRQ_FLAG_FLOATING ) + pending_irqs |= vcpu->kvm->arch.float_int.pending_irqs; +if (irq_flags &IRQ_FLAG_GISA )
Fix crash under vsie:
pending_irqs |= vcpu->kvm->arch.float_int.pending_irqs;
- if (irq_flags & IRQ_FLAG_GISA)
+ if (vcpu->kvm->arch.gisa && irq_flags & IRQ_FLAG_GISA)
pending_irqs |= get_ipm(vcpu->kvm->arch.gisa, irq_flags)Quoted 4 lines
arch/s390/kvm/interrupt.c +3
-0+ pending_irqs |=kvm_s390_gisa_get_ipm (vcpu->kvm->arch.gisa) << +IRQ_PEND_IO_ISC_7 ; +return pending_irqs; }
Signature · 8 lines
Mit freundlichen Grüßen / Kind regards Michael Müller IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Matthias Hartmann Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen Registergericht: Amtsgericht Stuttgart, HRB 243294
Re: [PATCH v5 05/15] KVM: s390: unify pending_irqs() and pending_irqs_no_gisa()
In reply to Michael MuellerQuoted 7 lines · On Wed, 19 Dec 2018 20:17:46 +0100 Michael Mueller <[email protected]> wrote:
Use a single function with parameter
irq_flagsto differentiate between cases.New irq flag:
IRQ_FLAG_LOCAL: include vcpu local interruptions pending
IRQ_FLAG_FLOATING: include vcpu floating interruptions pending
IRQ_FLAG_GISA: include GISA interruptions pending in IPM
I presume that means that irqs may be in more than one set? Or are gisa irqs not considered floating irqs, because they use a different mechanism?
Quoted 35 lines
New irq masks:
IRQ_MASK_ALL: include all types
IRQ_MASK_NO_GISA: include all types but GISAExamples:
pending_irqs(vcpu, IRQ_MASK_ALL)
pending_irqs(vcpu, IRQ_MASK_NO_GISA)There will be more irq flags with upcoming patches.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 33 +++++++++++++++++++++------------ 1 file changed, 21 insertions(+), 12 deletions(-)arch/s390/kvm/interrupt.c +7
-0@@ -31,6 +31,13 @@#define PFAULT_DONE 0x0680#define VIRTIO_PARAM 0x0d00 +#define IRQ_FLAG_LOCAL 0x8000/* include local interruption pending mask */ +#define IRQ_FLAG_FLOATING 0x4000/* include float interruption pending mask */ +#define IRQ_FLAG_GISA 0x2000/* include GISA interruption pending mask */ + +#define IRQ_MASK_ALL (IRQ_FLAG_LOCAL | IRQ_FLAG_FLOATING | IRQ_FLAG_GISA) +#define IRQ_MASK_NO_GISA (IRQ_MASK_ALL & ~IRQ_FLAG_GISA) +/* handle external calls via sigp interpretation facility */ static int sca_ext_call_pending (struct kvm_vcpu *vcpu,int *src_id) {
arch/s390/kvm/interrupt.c static inline int kvm_s390_gisa_tac_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gis · line 247
Quoted 5 lines
arch/s390/kvm/interrupt.c +1
-1@@ -237,16 +244,18 @@ static inline int kvm_s390_gisa_tac_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gisreturn test_and_clear_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); } -static inline unsigned long pending_irqs_no_gisa (struct kvm_vcpu *vcpu) +static inline unsigned long pending_irqs (struct kvm_vcpu *vcpu,u16 irq_flags)
Any deeper reason why this is a u16? 16 bits should be enough for everyone? :)
Quoted 31 lines
arch/s390/kvm/interrupt.c +10
-8{ -return vcpu->kvm->arch.float_int.pending_irqs | - vcpu->arch.local_int.pending_irqs; -} +unsigned long pending_irqs =0 ; -static inline unsigned long pending_irqs (struct kvm_vcpu *vcpu) -{ -return pending_irqs_no_gisa (vcpu) | -kvm_s390_gisa_get_ipm (vcpu->kvm->arch.gisa) <<IRQ_PEND_IO_ISC_7 ; +if (irq_flags &IRQ_FLAG_LOCAL ) + pending_irqs |= vcpu->arch.local_int.pending_irqs; +if (irq_flags &IRQ_FLAG_FLOATING ) + pending_irqs |= vcpu->kvm->arch.float_int.pending_irqs; +if (irq_flags &IRQ_FLAG_GISA ) + pending_irqs |=kvm_s390_gisa_get_ipm (vcpu->kvm->arch.gisa) << +IRQ_PEND_IO_ISC_7 ; +return pending_irqs; }static inline int isc_to_irq_type (unsigned long isc)@@ -275,7 +284,7 @@ static unsigned long deliverable_irqs(struct kvm_vcpu *vcpu) {unsigned long active_mask; - active_mask =pending_irqs (vcpu); + active_mask =pending_irqs (vcpu,IRQ_MASK_ALL );if (!active_mask)return 0 ;
arch/s390/kvm/interrupt.c static void __reset_intercept_indicators(struct kvm_vcpu *vcpu) · line 355
Quoted 5 lines
arch/s390/kvm/interrupt.c +1
-1@@ -343,7 +352,7 @@ static void __reset_intercept_indicators(struct kvm_vcpu *vcpu)static void set_intercept_indicators_io (struct kvm_vcpu *vcpu) { -if (!(pending_irqs_no_gisa (vcpu) &IRQ_PEND_IO_MASK )) +if (!(pending_irqs (vcpu,IRQ_MASK_NO_GISA ) &IRQ_PEND_IO_MASK ))
I/O interrupts are always floating, so you probably could check for only floating (excluding gisa) irqs here.
Quoted 21 lines
arch/s390/kvm/interrupt.c +2
-2return ; else if (psw_ioint_disabled(vcpu)) kvm_s390_set_cpuflags(vcpu, CPUSTAT_IO_INT);Store Data@@ -353,7 +362,7 @@ static void set_intercept_indicators_io(struct kvm_vcpu *vcpu)static void set_intercept_indicators_ext (struct kvm_vcpu *vcpu) { -if (!(pending_irqs_no_gisa (vcpu) &IRQ_PEND_EXT_MASK )) +if (!(pending_irqs (vcpu,IRQ_MASK_NO_GISA ) &IRQ_PEND_EXT_MASK ))return ;if (psw_extint_disabled (vcpu))kvm_s390_set_cpuflags (vcpu,CPUSTAT_EXT_INT );@@ -363,7 +372,7 @@ static void set_intercept_indicators_ext(struct kvm_vcpu *vcpu)static void set_intercept_indicators_mchk (struct kvm_vcpu *vcpu) { -if (!(pending_irqs_no_gisa (vcpu) &IRQ_PEND_MCHK_MASK )) +if (!(pending_irqs (vcpu,IRQ_MASK_NO_GISA ) &IRQ_PEND_MCHK_MASK ))return ;if (psw_mchk_disabled (vcpu)) vcpu->arch.sie_block->ictl |=ICTL_LPSW ;
Re: [PATCH v5 05/15] KVM: s390: unify pending_irqs() and pending_irqs_no_gisa()
Quoted 6 lines · Cornelia Huck
Quoted 3 lines · On Wed, 19 Dec 2018 20:17:46 +0100 Michael Mueller <[email protected]> wrote:
Use a single function with parameter
irq_flagsto differentiate between cases.
Quoted 8 lines
Quoted 4 lines
New irq flag:
IRQ_FLAG_LOCAL: include vcpu local interruptions pending
IRQ_FLAG_FLOATING: include vcpu floating interruptions pending
IRQ_FLAG_GISA: include GISA interruptions pending in IPMI presume that means that irqs may be in more than one set? Or are gisa irqs not considered floating irqs, because they use a different mechanism?
Currently, the interruptions managed in GISA are floating only. But
that might change in future. The idea is not to subsume IRQ_FLAG_GISA
in IRQ_FLAG_FLOATING but to be able to address the right set of procedures to determine the irq pending set for a given subset of irq types that have different implementations.
There might be a better name for IRQ_FLAG_FLOATING then?
Quoted 39 lines
Quoted 39 lines
New irq masks:
IRQ_MASK_ALL: include all types
IRQ_MASK_NO_GISA: include all types but GISAExamples:
pending_irqs(vcpu, IRQ_MASK_ALL)
pending_irqs(vcpu, IRQ_MASK_NO_GISA)There will be more irq flags with upcoming patches.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 33 +++++++++++++++++++++------------ 1 file changed, 21 insertions(+), 12 deletions(-)arch/s390/kvm/interrupt.c +7
-0@@ -31,6 +31,13 @@#define PFAULT_DONE 0x0680#define VIRTIO_PARAM 0x0d00 +#define IRQ_FLAG_LOCAL 0x8000/* include local interruption pending mask */ +#define IRQ_FLAG_FLOATING 0x4000/* include float interruption pending mask */ +#define IRQ_FLAG_GISA 0x2000/* include GISA interruption pending mask */ + +#define IRQ_MASK_ALL (IRQ_FLAG_LOCAL | IRQ_FLAG_FLOATING | IRQ_FLAG_GISA) +#define IRQ_MASK_NO_GISA (IRQ_MASK_ALL & ~IRQ_FLAG_GISA) +/* handle external calls via sigp interpretation facility */ static int sca_ext_call_pending (struct kvm_vcpu *vcpu,int *src_id) {@@ -237,16 +244,18 @@ static inline int kvm_s390_gisa_tac_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gisreturn test_and_clear_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); }
Quoted 5 lines
arch/s390/kvm/interrupt.c static inline int kvm_s390_gisa_tac_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gis · line 247
Quoted 2 lines
arch/s390/kvm/interrupt.c +1
-1-static inline unsigned long pending_irqs_no_gisa (struct kvm_vcpu *vcpu) +static inline unsigned long pending_irqs (struct kvm_vcpu *vcpu,u16 irq_flags)Any deeper reason why this is a u16? 16 bits should be enough for everyone? :)
I want to use the 8 bits for the IRQ type and the other 8 for additional controls, see: "KVM: s390: restore IAM in get_ipm() when IPM is clean"
Quoted 33 lines
Quoted 33 lines
arch/s390/kvm/interrupt.c +10
-8{ -return vcpu->kvm->arch.float_int.pending_irqs | - vcpu->arch.local_int.pending_irqs; -} +unsigned long pending_irqs =0 ; -static inline unsigned long pending_irqs (struct kvm_vcpu *vcpu) -{ -return pending_irqs_no_gisa (vcpu) | -kvm_s390_gisa_get_ipm (vcpu->kvm->arch.gisa) <<IRQ_PEND_IO_ISC_7 ; +if (irq_flags &IRQ_FLAG_LOCAL ) + pending_irqs |= vcpu->arch.local_int.pending_irqs; +if (irq_flags &IRQ_FLAG_FLOATING ) + pending_irqs |= vcpu->kvm->arch.float_int.pending_irqs; +if (irq_flags &IRQ_FLAG_GISA ) + pending_irqs |=kvm_s390_gisa_get_ipm (vcpu->kvm->arch.gisa) << +IRQ_PEND_IO_ISC_7 ; +return pending_irqs; }static inline int isc_to_irq_type (unsigned long isc)@@ -275,7 +284,7 @@ static unsigned long deliverable_irqs(struct kvm_vcpu *vcpu) {unsigned long active_mask; - active_mask =pending_irqs (vcpu); + active_mask =pending_irqs (vcpu,IRQ_MASK_ALL );if (!active_mask)return 0 ;@@ -343,7 +352,7 @@ static void __reset_intercept_indicators(struct kvm_vcpu *vcpu)
Quoted 7 lines
arch/s390/kvm/interrupt.c static void __reset_intercept_indicators(struct kvm_vcpu *vcpu) · line 355
Quoted 4 lines
arch/s390/kvm/interrupt.c +1
-1static void set_intercept_indicators_io (struct kvm_vcpu *vcpu) { -if (!(pending_irqs_no_gisa (vcpu) &IRQ_PEND_IO_MASK )) +if (!(pending_irqs (vcpu,IRQ_MASK_NO_GISA ) &IRQ_PEND_IO_MASK ))I/O interrupts are always floating, so you probably could check for only floating (excluding gisa) irqs here.
That's right.
Quoted 6 lines
Quoted 6 lines
arch/s390/kvm/interrupt.c +0
-0return ; else if (psw_ioint_disabled(vcpu)) kvm_s390_set_cpuflags(vcpu, CPUSTAT_IO_INT);Store Data@@ -353,7 +362,7 @@ static void set_intercept_indicators_io(struct kvm_vcpu *vcpu)
Re: [PATCH v5 05/15] KVM: s390: unify pending_irqs() and pending_irqs_no_gisa()
Quoted 14 lines · On Thu, 20 Dec 2018 12:49:56 +0100 Michael Mueller <[email protected]> wrote:
Quoted 11 lines · Cornelia Huck
Quoted 7 lines · On Wed, 19 Dec 2018 20:17:46 +0100 Michael Mueller <[email protected]> wrote:
Use a single function with parameter
irq_flagsto differentiate between cases.New irq flag:
IRQ_FLAG_LOCAL: include vcpu local interruptions pending
IRQ_FLAG_FLOATING: include vcpu floating interruptions pending
IRQ_FLAG_GISA: include GISA interruptions pending in IPM
Quoted 6 lines
Quoted 3 lines
I presume that means that irqs may be in more than one set? Or are gisa irqs not considered floating irqs, because they use a different mechanism?
Currently, the interruptions managed in GISA are floating only. But that might change in future.
I don't think GISA can be used for non-floating interrupts.
Regards,
Halil
Quoted 7 lines
The idea is not to subsume
IRQ_FLAG_GISA
inIRQ_FLAG_FLOATINGbut to be able to address the right set of procedures to determine the irq pending set for a given subset of irq types that have different implementations.There might be a better name for
IRQ_FLAG_FLOATINGthen?
[..]
Re: [PATCH v5 05/15] KVM: s390: unify pending_irqs() and pending_irqs_no_gisa()
In reply to Michael MuellerQuoted 16 lines · On Thu, 20 Dec 2018 12:49:56 +0100 Michael Mueller <[email protected]> wrote:
Quoted 6 lines · Cornelia Huck
Quoted 3 lines · On Wed, 19 Dec 2018 20:17:46 +0100 Michael Mueller <[email protected]> wrote:
Use a single function with parameter
irq_flagsto differentiate between cases.Quoted 8 lines
Quoted 4 lines
New irq flag:
IRQ_FLAG_LOCAL: include vcpu local interruptions pending
IRQ_FLAG_FLOATING: include vcpu floating interruptions pending
IRQ_FLAG_GISA: include GISA interruptions pending in IPMI presume that means that irqs may be in more than one set? Or are gisa irqs not considered floating irqs, because they use a different mechanism?
Quoted 7 lines
Currently, the interruptions managed in GISA are floating only. But that might change in future. The idea is not to subsume
IRQ_FLAG_GISAinIRQ_FLAG_FLOATINGbut to be able to address the right set of procedures to determine the irq pending set for a given subset of irq types that have different implementations.There might be a better name for
IRQ_FLAG_FLOATINGthen?
So the split is:
- local interrupts that are pending via kvm structures;
- floating interrupts that are pending via kvm structures;
- interrupts that are pending via gisa?
If so, what about
- IRQ_FLAG_KVM_LOCAL
- IRQ_FLAG_KVM_FLOATING
- IRQ_FLAG_GISA (or maybe IRQ_FLAG_GISA_FLOATING, if you need to distinguish those later on?)
Quoted 40 lines
Quoted 40 lines
Quoted 40 lines
New irq masks:
IRQ_MASK_ALL: include all types
IRQ_MASK_NO_GISA: include all types but GISAExamples:
pending_irqs(vcpu, IRQ_MASK_ALL)
pending_irqs(vcpu, IRQ_MASK_NO_GISA)There will be more irq flags with upcoming patches.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 33 +++++++++++++++++++++------------ 1 file changed, 21 insertions(+), 12 deletions(-)arch/s390/kvm/interrupt.c +7
-0@@ -31,6 +31,13 @@#define PFAULT_DONE 0x0680#define VIRTIO_PARAM 0x0d00 +#define IRQ_FLAG_LOCAL 0x8000/* include local interruption pending mask */ +#define IRQ_FLAG_FLOATING 0x4000/* include float interruption pending mask */ +#define IRQ_FLAG_GISA 0x2000/* include GISA interruption pending mask */ + +#define IRQ_MASK_ALL (IRQ_FLAG_LOCAL | IRQ_FLAG_FLOATING | IRQ_FLAG_GISA) +#define IRQ_MASK_NO_GISA (IRQ_MASK_ALL & ~IRQ_FLAG_GISA) +/* handle external calls via sigp interpretation facility */ static int sca_ext_call_pending (struct kvm_vcpu *vcpu,int *src_id) {@@ -237,16 +244,18 @@ static inline int kvm_s390_gisa_tac_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gisreturn test_and_clear_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); }
Quoted 8 lines
Quoted 5 lines
arch/s390/kvm/interrupt.c static inline int kvm_s390_gisa_tac_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gis · line 247
Quoted 2 lines
arch/s390/kvm/interrupt.c +1
-1-static inline unsigned long pending_irqs_no_gisa (struct kvm_vcpu *vcpu) +static inline unsigned long pending_irqs (struct kvm_vcpu *vcpu,u16 irq_flags)Any deeper reason why this is a u16? 16 bits should be enough for everyone? :)
I want to use the 8 bits for the IRQ type and the other 8 for additional controls, see: "KVM: s390: restore IAM in
get_ipm()when IPM is clean"
Still need to look at that patch, but my question mainly was "why only 16 bits"? I would think making this local variable larger is cheap.
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
arch/s390/kvm/interrupt.c +0
-3{ -return vcpu->kvm->arch.float_int.pending_irqs | - vcpu->arch.local_int.pending_irqs; -}
Re: [PATCH v5 05/15] KVM: s390: unify pending_irqs() and pending_irqs_no_gisa()
Quoted 33 lines · Cornelia Huck
Quoted 16 lines · On Thu, 20 Dec 2018 12:49:56 +0100 Michael Mueller <[email protected]> wrote:
Quoted 6 lines · Cornelia Huck
Quoted 3 lines · On Wed, 19 Dec 2018 20:17:46 +0100 Michael Mueller <[email protected]> wrote:
Use a single function with parameter
irq_flagsto differentiate between cases.Quoted 8 lines
Quoted 4 lines
New irq flag:
IRQ_FLAG_LOCAL: include vcpu local interruptions pending
IRQ_FLAG_FLOATING: include vcpu floating interruptions pending
IRQ_FLAG_GISA: include GISA interruptions pending in IPMI presume that means that irqs may be in more than one set? Or are gisa irqs not considered floating irqs, because they use a different mechanism?
Quoted 7 lines
Currently, the interruptions managed in GISA are floating only. But that might change in future. The idea is not to subsume
IRQ_FLAG_GISAinIRQ_FLAG_FLOATINGbut to be able to address the right set of procedures to determine the irq pending set for a given subset of irq types that have different implementations.There might be a better name for
IRQ_FLAG_FLOATINGthen?So the split is:
- local interrupts that are pending via kvm structures;
- floating interrupts that are pending via kvm structures;
- interrupts that are pending via gisa?
Quoted 5 lines
If so, what about
-IRQ_FLAG_KVM_LOCAL
-IRQ_FLAG_KVM_FLOATING
-IRQ_FLAG_GISA(or maybeIRQ_FLAG_GISA_FLOATING, if you need to distinguish those later on?)
yes, that's the split and I will go with:
IRQ_FLAG_KVM_LOCALIRQ_FLAG_KVM_FLOATINGIRQ_FLAG_GISA
initially.
The floating and local selection can be done by related masks
IRQ_MASK_LOCALIRQ_MASK_FLOATING
if required.
Thanks!
Quoted 47 lines
Quoted 47 lines
Quoted 46 lines
Quoted 38 lines
New irq masks:
IRQ_MASK_ALL: include all types
IRQ_MASK_NO_GISA: include all types but GISAExamples:
pending_irqs(vcpu, IRQ_MASK_ALL)
pending_irqs(vcpu, IRQ_MASK_NO_GISA)There will be more irq flags with upcoming patches.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 33 +++++++++++++++++++++------------ 1 file changed, 21 insertions(+), 12 deletions(-)arch/s390/kvm/interrupt.c +7
-0@@ -31,6 +31,13 @@#define PFAULT_DONE 0x0680#define VIRTIO_PARAM 0x0d00 +#define IRQ_FLAG_LOCAL 0x8000/* include local interruption pending mask */ +#define IRQ_FLAG_FLOATING 0x4000/* include float interruption pending mask */ +#define IRQ_FLAG_GISA 0x2000/* include GISA interruption pending mask */ + +#define IRQ_MASK_ALL (IRQ_FLAG_LOCAL | IRQ_FLAG_FLOATING | IRQ_FLAG_GISA) +#define IRQ_MASK_NO_GISA (IRQ_MASK_ALL & ~IRQ_FLAG_GISA) +/* handle external calls via sigp interpretation facility */ static int sca_ext_call_pending (struct kvm_vcpu *vcpu,int *src_id) {arch/s390/kvm/interrupt.c static inline int kvm_s390_gisa_tac_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gis · line 247
Quoted 5 lines
arch/s390/kvm/interrupt.c +1
-1@@ -237,16 +244,18 @@ static inline int kvm_s390_gisa_tac_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gisreturn test_and_clear_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); } -static inline unsigned long pending_irqs_no_gisa (struct kvm_vcpu *vcpu) +static inline unsigned long pending_irqs (struct kvm_vcpu *vcpu,u16 irq_flags)Any deeper reason why this is a u16? 16 bits should be enough for everyone? :)
Quoted 6 lines
Quoted 2 lines
I want to use the 8 bits for the IRQ type and the other 8 for additional controls, see: "KVM: s390: restore IAM in
get_ipm()when IPM is clean"Still need to look at that patch, but my question mainly was "why only 16 bits"? I would think making this local variable larger is cheap.
I will enlarge the flag mask to u32 with 16 bits for the IRQ types then.
Quoted 7 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
arch/s390/kvm/interrupt.c +0
-3{ -return vcpu->kvm->arch.float_int.pending_irqs | - vcpu->arch.local_int.pending_irqs; -}
Signature · 8 lines
Mit freundlichen Grüßen / Kind regards Michael Müller IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Matthias Hartmann Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen Registergericht: Amtsgericht Stuttgart, HRB 243294
Re: [PATCH v5 05/15] KVM: s390: unify pending_irqs() and pending_irqs_no_gisa()
Le 12/20/18 à 13:33, Michael Mueller a écrit :
Quoted 12 lines · Cornelia Huck
Quoted 9 lines
Quoted 6 lines · On Thu, 20 Dec 2018 12:49:56 +0100 Michael Mueller <[email protected]> wrote:
Quoted 5 lines · Cornelia Huck
Quoted 3 lines · On Wed, 19 Dec 2018 20:17:46 +0100 Michael Mueller <[email protected]> wrote:
Use a single function with parameter
irq_flagsto differentiate between cases.
...snip
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 5 lines
}
-static inline unsigned longpending_irqs_no_gisa(struct kvm_vcpu *vcpu)
+static inline unsigned longpending_irqs(struct kvm_vcpu *vcpu, u16 irq_flags)Any deeper reason why this is a u16? 16 bits should be enough for everyone? :)
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
I want to use the 8 bits for the IRQ type and the other 8 for additional controls, see: "KVM: s390: restore IAM in
get_ipm()when IPM is clean"Still need to look at that patch, but my question mainly was "why only 16 bits"? I would think making this local variable larger is cheap.
+1
Quoted 2 lines
I will enlarge the flag mask to u32 with 16 bits for the IRQ types then.
AFAIK CPU generally work better with int (or long) Is there any hardware reason to restrict the size?
Quoted 8 lines
Quoted 7 lines
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
{
- returnvcpu->kvm->arch.float_int.pending_irqs| -vcpu->arch.local_int.pending_irqs; -}
Re: [PATCH v5 05/15] KVM: s390: unify pending_irqs() and pending_irqs_no_gisa()
Quoted 32 lines · pierre morel
Le 12/20/18 à 13:33, Michael Mueller a écrit :
Quoted 12 lines · Cornelia Huck
Quoted 9 lines
Quoted 6 lines · On Thu, 20 Dec 2018 12:49:56 +0100 Michael Mueller <[email protected]> wrote:
Quoted 5 lines · Cornelia Huck
Quoted 3 lines · On Wed, 19 Dec 2018 20:17:46 +0100 Michael Mueller <[email protected]> wrote:
Use a single function with parameter
irq_flagsto differentiate between cases....snip
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 5 lines
}
-static inline unsigned longpending_irqs_no_gisa(struct kvm_vcpu *vcpu)
+static inline unsigned longpending_irqs(struct kvm_vcpu *vcpu, u16 irq_flags)Any deeper reason why this is a u16? 16 bits should be enough for everyone? :)
Quoted 6 lines
Quoted 6 lines
Quoted 2 lines
I want to use the 8 bits for the IRQ type and the other 8 for additional controls, see: "KVM: s390: restore IAM in
get_ipm()when IPM is clean"Still need to look at that patch, but my question mainly was "why only 16 bits"? I would think making this local variable larger is cheap.
Quoted 7 lines
+1
Quoted 2 lines
I will enlarge the flag mask to u32 with 16 bits for the IRQ types then.
AFAIK CPU generally work better with int (or long) Is there any hardware reason to restrict the size?
It's already changed to 4 bytes
Quoted 10 lines
Quoted 9 lines
Quoted 8 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
{
- returnvcpu->kvm->arch.float_int.pending_irqs| -vcpu->arch.local_int.pending_irqs; -}
PATCH v5 6/15 KVM: s390: remove prefix kvm_s390_gisa_ from static inline functions
In reply to Michael MuellerThis will shorten the length of code lines.
All GISA related static inline functions are local to interrupt.c
--- arch/s390/kvm/interrupt.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)
arch/s390/kvm/interrupt.c +13 -13
@@ -224,22 +224,22 @@ static inline u8 int_word_to_isc(u32 int_word)*/ #define IPM_BIT_OFFSET (offsetof(struct kvm_s390_gisa, ipm) * BITS_PER_BYTE) -static inline void kvm_s390_gisa_set_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) +static inline void set_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) {set_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); } -static inline u8 kvm_s390_gisa_get_ipm (struct kvm_s390_gisa *gisa) +static inline u8 get_ipm (struct kvm_s390_gisa *gisa) {return READ_ONCE (gisa->ipm); } -static inline void kvm_s390_gisa_clear_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) +static inline void clear_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) {clear_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); } -static inline int kvm_s390_gisa_tac_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) +static inline int tac_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) {return test_and_clear_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); }
@@ -253,7 +253,7 @@ static inline unsigned long pending_irqs(struct kvm_vcpu *vcpu, u16 irq_flags)if (irq_flags &IRQ_FLAG_FLOATING ) pending_irqs |= vcpu->kvm->arch.float_int.pending_irqs;if (irq_flags &IRQ_FLAG_GISA ) - pending_irqs |=kvm_s390_gisa_get_ipm (vcpu->kvm->arch.gisa) << + pending_irqs |=get_ipm (vcpu->kvm->arch.gisa) <<IRQ_PEND_IO_ISC_7 ;return pending_irqs; }
@@ -1008,7 +1008,7 @@ static int __must_check __deliver_io(struct kvm_vcpu *vcpu, }if (vcpu->kvm->arch.gisa && -kvm_s390_gisa_tac_ipm_gisc (vcpu->kvm->arch.gisa, isc)) { +tac_ipm_gisc (vcpu->kvm->arch.gisa, isc)) {/* * in case an adapter interrupt was not delivered * in SIE context KVM will handle the delivery
@@ -1550,10 +1550,10 @@ static int get_top_gisa_isc(struct kvm *kvm, u64 isc_mask, u32 schid)if (!kvm->arch.gisa) goto out; - active_mask = (isc_mask &kvm_s390_gisa_get_ipm (kvm->arch.gisa) <<24 ) <<32 ; + active_mask = (isc_mask &get_ipm (kvm->arch.gisa) <<24 ) <<32 ;while (active_mask) { isc =__fls (active_mask) ^ (BITS_PER_LONG -1 ); -if (kvm_s390_gisa_tac_ipm_gisc (kvm->arch.gisa, isc)) +if (tac_ipm_gisc (kvm->arch.gisa, isc))return isc;clear_bit_inv (isc, &active_mask); }
@@ -1593,7 +1593,7 @@ struct kvm_s390_interrupt_info *kvm_s390_get_io_int(struct kvm *kvm,/* both types of interrupts present */ if (int_word_to_isc (inti->io.io_int_word) <= isc) {/* classical IO int with higher priority */ -kvm_s390_gisa_set_ipm_gisc (kvm->arch.gisa, isc); +set_ipm_gisc (kvm->arch.gisa, isc); goto out; } gisa_out:
@@ -1605,7 +1605,7 @@ struct kvm_s390_interrupt_info *kvm_s390_get_io_int(struct kvm *kvm,kvm_s390_reinject_io_int (kvm, inti); inti = tmp_inti; }else -kvm_s390_gisa_set_ipm_gisc (kvm->arch.gisa, isc); +set_ipm_gisc (kvm->arch.gisa, isc); out:return inti; }
@@ -1703,7 +1703,7 @@ static int __inject_io(struct kvm *kvm, struct kvm_s390_interrupt_info *inti)if (kvm->arch.gisa && inti->type &KVM_S390_INT_IO_AI_MASK ) {VM_EVENT (kvm,4 ,"%s isc %1u" ,"inject: I/O (AI/gisa)" , isc); -kvm_s390_gisa_set_ipm_gisc (kvm->arch.gisa, isc); +set_ipm_gisc (kvm->arch.gisa, isc);kfree (inti);return 0 ; }
@@ -2036,14 +2036,14 @@ static int get_all_floating_irqs(struct kvm *kvm, u8 __user *usrbuf, u64 len) max_irqs = len /sizeof (struct kvm_s390_irq );if (kvm->arch.gisa && -kvm_s390_gisa_get_ipm (kvm->arch.gisa)) { +get_ipm (kvm->arch.gisa)) {for (i =0 ; i <=MAX_ISC ; i++) {if (n == max_irqs) {/* signal userspace to try again */ ret = -ENOMEM ; goto out_nolock; } -if (kvm_s390_gisa_tac_ipm_gisc (kvm->arch.gisa, i)) { +if (tac_ipm_gisc (kvm->arch.gisa, i)) { irq = (struct kvm_s390_irq *) &buf[n]; irq->type =KVM_S390_INT_IO (1 ,0 ,0 ,0 ); irq->u.io.io_int_word =isc_to_int_word (i);
Signature · 1 line
2.13.4
Re: [PATCH v5 06/15] KVM: s390: remove prefix kvm_s390_gisa_ from static inline functions
Quoted 29 lines · On Wed, 19 Dec 2018 20:17:47 +0100 Michael Mueller <[email protected]> wrote:
This will shorten the length of code lines.
All GISA related static inline functions are local tointerrupt.cSigned-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)arch/s390/kvm/interrupt.c +2
-3@@ -224,22 +224,22 @@ static inline u8 int_word_to_isc(u32 int_word)*/ #define IPM_BIT_OFFSET (offsetof(struct kvm_s390_gisa, ipm) * BITS_PER_BYTE) -static inline void kvm_s390_gisa_set_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) +static inline void set_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) {set_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); } -static inline u8 kvm_s390_gisa_get_ipm (struct kvm_s390_gisa *gisa) +static inline u8 get_ipm (struct kvm_s390_gisa *gisa) {return READ_ONCE (gisa->ipm); } -static inline void kvm_s390_gisa_clear_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc)
arch/s390/kvm/interrupt.c static inline u8 int_word_to_isc(u32 int_word) · line 245
Quoted 10 lines
arch/s390/kvm/interrupt.c +2
-1+static inline void clear_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) {clear_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); } -static inline int kvm_s390_gisa_tac_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) +static inline int tac_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) {return test_and_clear_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); }
I don't disagree with making them shorter, but I think the code would be more readable if you only dropped the kvm_s390_ prefix and kept annotating the functions as gisa_.
Michael Mueller
AppliedRe: [PATCH v5 06/15] KVM: s390: remove prefix kvm_s390_gisa_ from static inline functions
Quoted 37 lines · Cornelia Huck
Quoted 34 lines · On Wed, 19 Dec 2018 20:17:47 +0100 Michael Mueller <[email protected]> wrote:
This will shorten the length of code lines.
All GISA related static inline functions are local tointerrupt.cSigned-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)arch/s390/kvm/interrupt.c +3
-3@@ -224,22 +224,22 @@ static inline u8 int_word_to_isc(u32 int_word)*/ #define IPM_BIT_OFFSET (offsetof(struct kvm_s390_gisa, ipm) * BITS_PER_BYTE) -static inline void kvm_s390_gisa_set_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) +static inline void set_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) {set_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); } -static inline u8 kvm_s390_gisa_get_ipm (struct kvm_s390_gisa *gisa) +static inline u8 get_ipm (struct kvm_s390_gisa *gisa) {return READ_ONCE (gisa->ipm); } -static inline void kvm_s390_gisa_clear_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) +static inline void clear_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) {clear_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); }
Quoted 9 lines
arch/s390/kvm/interrupt.c static inline u8 int_word_to_isc(u32 int_word) · line 245
Quoted 5 lines
arch/s390/kvm/interrupt.c +1
-1-static inline int kvm_s390_gisa_tac_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) +static inline int tac_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) {return test_and_clear_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); }I don't disagree with making them shorter, but I think the code would be more readable if you only dropped the
kvm_s390_prefix and kept annotating the functions asgisa_.
applied
Quoted 1 line
PATCH v5 7/15 s390/cio: add function chsc_sgib()
In reply to Michael MuellerThis patch implements the Set Guest Information Block operation to request association or disassociation of a Guest Information Block (GIB) with the Adapter Interruption Facility. The operation is required to receive GIB alert interrupts for guest adapters in conjunction with AIV and GISA.
--- arch/s390/include/asm/cio.h | 1 + drivers/s390/cio/chsc.c | 37 +++++++++++++++++++++++++++++++++++++ drivers/s390/cio/chsc.h | 1 + 3 files changed, 39 insertions(+)
arch/s390/include/asm/cio.h +1 -0
@@ -331,5 +331,6 @@ extern void css_schedule_reprobe(void);/* Function from drivers/s390/cio/chsc.c */ int chsc_sstpc (void *page,unsigned int op,u16 ctrl,u64 *clock_delta);int chsc_sstpi (void *page,void *result,size_t size); +int chsc_sgib (u32 origin);#endif
drivers/s390/cio/chsc.c +37 -0
@@ -1382,3 +1382,40 @@ int chsc_pnso_brinfo(struct subchannel_id schid,return chsc_error_from_response (brinfo_area->response.code); }EXPORT_SYMBOL_GPL (chsc_pnso_brinfo); + +int chsc_sgib (u32 origin) +{ +struct { +struct chsc_header request; +u16 op; +u8 reserved01[2 ]; +u8 reserved02:4 ; +u8 fmt:4 ; +u8 reserved03[7 ]; +/* operation data area begin */ +u8 reserved04[4 ]; +u32 gib_origin; +u8 reserved05[10 ]; +u8 aix; +u8 reserved06[4029 ]; +struct chsc_header response; +u8 reserved07[4 ]; + } *sgib_area; +int ret; + +spin_lock_irq (&chsc_page_lock); +memset (chsc_page,0 ,PAGE_SIZE ); + sgib_area = chsc_page; + sgib_area->request.length =0x0fe0 ; + sgib_area->request.code =0x0021 ; + sgib_area->op =0x1 ; + sgib_area->gib_origin = origin; + + ret =chsc (sgib_area); +if (ret ==0 ) + ret =chsc_error_from_response (sgib_area->response.code); +spin_unlock_irq (&chsc_page_lock); + +return ret; +} +EXPORT_SYMBOL_GPL (chsc_sgib);
drivers/s390/cio/chsc.h +1 -0
@@ -164,6 +164,7 @@ int chsc_get_channel_measurement_chars(struct channel_path *chp);int chsc_ssqd (struct subchannel_id schid,struct chsc_ssqd_area *ssqd);int chsc_sadc (struct subchannel_id schid,struct chsc_scssc_area *scssc,u64 summary_indicator_addr,u64 subchannel_indicator_addr); +int chsc_sgib (u32 origin);int chsc_error_from_response (int response);int chsc_siosl (struct subchannel_id schid);
Signature · 1 line
2.13.4
PATCH v5 8/15 KVM: s390: add the GIB and its related life-cyle functions
In reply to Michael MuellerThe Guest Information Block (GIB) links the GISA of all guests that have adapter interrupts pending. These interrupts cannot be delivered because no vcpu of these guests is currently running in SIE context. Instead, a GIB alert is issued on the host to schedule these guests to run.
This mechanism allows to process adapter interrupts for currently not running guests.
The GIB is created during host initialization and associated with the Adapter Interruption Facility in case an Adapter Interruption Virtualization Facility is available.
The GIB initialization and thus the activation of the related code will be done in an upcoming patch of this series.
--- arch/s390/include/asm/kvm_host.h | 10 +++++++++ arch/s390/kvm/interrupt.c | 44 ++++++++++++++++++++++++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 1 + arch/s390/kvm/kvm-s390.h | 2 ++ 4 files changed, 57 insertions(+)
arch/s390/include/asm/kvm_host.h +10 -0
@@ -785,6 +785,15 @@ struct kvm_s390_gisa { }; }; +struct kvm_s390_gib { +u32 alert_list_origin; +u32 reserved01; + u8:5; +u8 nisc:3 ; +u8 reserved03[3 ]; +u32 reserved04[5 ]; +}; +/* * sie_page2 has to be allocated as DMA because fac_list, crycb and * gisa need 31bit addresses in the sie control block.
@@ -838,6 +847,7 @@ struct kvm_arch{/* subset of available cpu features enabled by user space */ DECLARE_BITMAP (cpu_feat,KVM_S390_VM_CPU_FEAT_NR_BITS );struct kvm_s390_gisa *gisa; +int gib_in_use; };#define KVM_HVA_ERR_BAD (-1UL)
arch/s390/kvm/interrupt.c +44 -0
65 lines not on this page · raw message
arch/s390/kvm/kvm-s390.c +1 -0
@@ -445,6 +445,7 @@ int kvm_arch_init(void *opaque)void kvm_arch_exit (void ) { +kvm_s390_gib_destroy ();debug_unregister (kvm_s390_dbf); }
arch/s390/kvm/kvm-s390.h +2 -0
@@ -381,6 +381,8 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu,void kvm_s390_gisa_init (struct kvm *kvm);void kvm_s390_gisa_clear (struct kvm *kvm);void kvm_s390_gisa_destroy (struct kvm *kvm); +int kvm_s390_gib_init (u8 nisc); +void kvm_s390_gib_destroy (void );/* implemented in guestdbg.c */ void kvm_s390_backup_guest_per_regs (struct kvm_vcpu *vcpu);
Signature · 1 line
2.13.4
Re: [PATCH v5 08/15] KVM: s390: add the GIB and its related life-cyle functions
Quoted 34 lines · On Wed, 19 Dec 2018 20:17:49 +0100 Michael Mueller <[email protected]> wrote:
The Guest Information Block (GIB) links the GISA of all guests that have adapter interrupts pending. These interrupts cannot be delivered because no vcpu of these guests is currently running in SIE context. Instead, a GIB alert is issued on the host to schedule these guests to run.
This mechanism allows to process adapter interrupts for currently not running guests.
The GIB is created during host initialization and associated with the Adapter Interruption Facility in case an Adapter Interruption Virtualization Facility is available.
The GIB initialization and thus the activation of the related code will be done in an upcoming patch of this series.
Signed-off-by Michael Mueller <[email protected]>Reviewed-by Janosch Frank <[email protected]>Reviewed-by Christian Borntraeger <[email protected]>--- arch/s390/include/asm/kvm_host.h | 10 +++++++++ arch/s390/kvm/interrupt.c | 44 ++++++++++++++++++++++++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 1 + arch/s390/kvm/kvm-s390.h | 2 ++ 4 files changed, 57 insertions(+)arch/s390/kvm/interrupt.c +8
-0+int kvm_s390_gib_init (u8 nisc) +{ +int rc =0 ; + +if (!css_general_characteristics.aiv) { +KVM_EVENT (3 ,"%s" ,"gib not initialized, no AIV facility" ); + goto out; + }
arch/s390/kvm/interrupt.c void kvm_s390_gisa_destroy(struct kvm *kvm) · line 2958
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+ + gib = (struct kvm_s390_gib *)get_zeroed_page (GFP_KERNEL |GFP_DMA ); +if (!gib) { + rc = -ENOMEM ; + goto out; + } + + gib->nisc = nisc; +if (chsc_sgib ((u32 )(u64 )gib)) { +pr_err ("Associating the GIB with the AIV facility failed\n" );
I presume that logging the chsc response code (or returning it instead of -EIO) would not give any interesting information?
arch/s390/kvm/interrupt.c void kvm_s390_gisa_destroy(struct kvm *kvm) · line 2968
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+free_page ((unsigned long )gib); + gib =NULL ; + rc = -EIO ; + goto out; + } + +KVM_EVENT (3 ,"gib 0x%pK (nisc=%d) initialized" , gib, gib->nisc); +out: +return rc; +}
Anyhow,
Re: [PATCH v5 08/15] KVM: s390: add the GIB and its related life-cyle functions
In reply to Michael MuellerQuoted 5 lines · Michael Mueller
The Guest Information Block (GIB) links the GISA of all guests that have adapter interrupts pending. These interrupts cannot be delivered because no vcpu of these guests is currently running in SIE context. Instead, a GIB alert is issued on the host to schedule these guests to run.
IMHO "Instead" is not exact.
The GIB alert is always issued on adapter interrupt if IAM is set even if the guest is running in SIE context.
I would prefer to rephrase to something like:
"If enabled, a GIB alert is issued on the host..." ?
Quoted 148 lines
This mechanism allows to process adapter interrupts for currently not running guests.
The GIB is created during host initialization and associated with the Adapter Interruption Facility in case an Adapter Interruption Virtualization Facility is available.
The GIB initialization and thus the activation of the related code will be done in an upcoming patch of this series.
Signed-off-by Michael Mueller <[email protected]>Reviewed-by Janosch Frank <[email protected]>Reviewed-by Christian Borntraeger <[email protected]>--- arch/s390/include/asm/kvm_host.h | 10 +++++++++ arch/s390/kvm/interrupt.c | 44 ++++++++++++++++++++++++++++++++++++++++ arch/s390/kvm/kvm-s390.c | 1 + arch/s390/kvm/kvm-s390.h | 2 ++ 4 files changed, 57 insertions(+)arch/s390/include/asm/kvm_host.h +10
-0@@ -785,6 +785,15 @@ struct kvm_s390_gisa { }; }; +struct kvm_s390_gib { +u32 alert_list_origin; +u32 reserved01; + u8:5; +u8 nisc:3 ; +u8 reserved03[3 ]; +u32 reserved04[5 ]; +}; +/* * sie_page2 has to be allocated as DMA because fac_list, crycb and * gisa need 31bit addresses in the sie control block. @@ -838,6 +847,7 @@ struct kvm_arch{/* subset of available cpu features enabled by user space */ DECLARE_BITMAP (cpu_feat,KVM_S390_VM_CPU_FEAT_NR_BITS );struct kvm_s390_gisa *gisa; +int gib_in_use; };#define KVM_HVA_ERR_BAD (-1UL)arch/s390/kvm/interrupt.c +44
-0@@ -7,6 +7,9 @@* Author(s): Carsten Otte <[email protected]> */ +#define KMSG_COMPONENT "kvm-s390" +#define pr_fmt (fmt) KMSG_COMPONENT ": " fmt +#include <linux/interrupt.h> #include <linux/kvm_host.h> #include <linux/hrtimer.h> @@ -38,6 +41,8 @@#define IRQ_MASK_ALL (IRQ_FLAG_LOCAL | IRQ_FLAG_FLOATING | IRQ_FLAG_GISA)#define IRQ_MASK_NO_GISA (IRQ_MASK_ALL & ~IRQ_FLAG_GISA) +static struct kvm_s390_gib *gib; +/* handle external calls via sigp interpretation facility */ static int sca_ext_call_pending (struct kvm_vcpu *vcpu,int *src_id) {@@ -2913,6 +2918,7 @@ void kvm_s390_gisa_init(struct kvm *kvm)return ; kvm->arch.gisa = &kvm->arch.sie_page2->gisa;nullify_gisa (kvm->arch.gisa); + kvm->arch.gib_in_use = !!gib;VM_EVENT (kvm,3 ,"gisa 0x%pK initialized" , kvm->arch.gisa); }@@ -2922,3 +2928,41 @@ void kvm_s390_gisa_destroy(struct kvm *kvm)return ; kvm->arch.gisa =NULL ; } + +void kvm_s390_gib_destroy (void ) +{ +if (!gib) +return ; +chsc_sgib (0 ); +free_page ((unsigned long )gib); + gib =NULL ; +} + +int kvm_s390_gib_init (u8 nisc) +{ +int rc =0 ; + +if (!css_general_characteristics.aiv) { +KVM_EVENT (3 ,"%s" ,"gib not initialized, no AIV facility" ); + goto out; + } + + gib = (struct kvm_s390_gib *)get_zeroed_page (GFP_KERNEL |GFP_DMA ); +if (!gib) { + rc = -ENOMEM ; + goto out; + } + + gib->nisc = nisc; +if (chsc_sgib ((u32 )(u64 )gib)) { +pr_err ("Associating the GIB with the AIV facility failed\n" ); +free_page ((unsigned long )gib); + gib =NULL ; + rc = -EIO ; + goto out; + } + +KVM_EVENT (3 ,"gib 0x%pK (nisc=%d) initialized" , gib, gib->nisc); +out: +return rc; +}arch/s390/kvm/kvm-s390.c +1
-0@@ -445,6 +445,7 @@ int kvm_arch_init(void *opaque)void kvm_arch_exit (void ) { +kvm_s390_gib_destroy ();debug_unregister (kvm_s390_dbf); }arch/s390/kvm/kvm-s390.h +2
-0@@ -381,6 +381,8 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu,void kvm_s390_gisa_init (struct kvm *kvm);void kvm_s390_gisa_clear (struct kvm *kvm);void kvm_s390_gisa_destroy (struct kvm *kvm); +int kvm_s390_gib_init (u8 nisc); +void kvm_s390_gib_destroy (void );/* implemented in guestdbg.c */ void kvm_s390_backup_guest_per_regs (struct kvm_vcpu *vcpu);
Signature · 2 lines
Pierre Morel Linux/KVM/QEMU in Böblingen - Germany
Re: [PATCH v5 08/15] KVM: s390: add the GIB and its related life-cyle functions
Quoted 7 lines · Pierre Morel
Quoted 5 lines · Michael Mueller
The Guest Information Block (GIB) links the GISA of all guests that have adapter interrupts pending. These interrupts cannot be delivered because no vcpu of these guests is currently running in SIE context. Instead, a GIB alert is issued on the host to schedule these guests to run.
Quoted 5 lines
IMHO "Instead" is not exact.
The GIB alert is always issued on adapter interrupt if IAM is set even if the guest is running in SIE context.I would prefer to rephrase to something like:
"If enabled, a GIB alert is issued on the host..." ?
I agree.
Quoted 21 lines
Reviewed-by Pierre Morel<[email protected]>Quoted 16 lines
This mechanism allows to process adapter interrupts for currently not running guests.
The GIB is created during host initialization and associated with the Adapter Interruption Facility in case an Adapter Interruption Virtualization Facility is available.
The GIB initialization and thus the activation of the related code will be done in an upcoming patch of this series.
Signed-off-by Michael Mueller <[email protected]>Reviewed-by Janosch Frank <[email protected]>Reviewed-by Christian Borntraeger <[email protected]>---
arch/s390/include/asm/kvm_host.h| 10 +++++++++
PATCH v5 9/15 KVM: s390: add kvm reference to struct sie_page2
In reply to Michael MuellerAdding the kvm reference to struct sie_page2 will allow to determine the kvm a given gisa belongs to:
container_of(gisa, struct sie_page2, gisa)->kvm
This functionality will be required to process a gisa in gib alert interruption context.
--- arch/s390/include/asm/kvm_host.h | 3 ++- arch/s390/kvm/kvm-s390.c | 1 + 2 files changed, 3 insertions(+), 1 deletion(-)
arch/s390/include/asm/kvm_host.h +2 -1
@@ -802,7 +802,8 @@ struct sie_page2 {__u64 fac_list[S390_ARCH_FAC_LIST_SIZE_U64 ];/* 0x0000 */ struct kvm_s390_crypto_cb crycb;/* 0x0800 */ struct kvm_s390_gisa gisa;/* 0x0900 */ -u8 reserved920[0x1000 -0x920 ];/* 0x0920 */ +struct kvm *kvm;/* 0x0920 */ +u8 reserved928[0x1000 -0x928 ];/* 0x0928 */ };struct kvm_s390_vsie {
arch/s390/kvm/kvm-s390.c +1 -0
@@ -2220,6 +2220,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)if (!kvm->arch.sie_page2) goto out_err; + kvm->arch.sie_page2->kvm = kvm; kvm->arch.model.fac_list = kvm->arch.sie_page2->fac_list;for (i =0 ; i <kvm_s390_fac_size (); i++) {
Signature · 1 line
2.13.4
PATCH v5 10/15 KVM: s390: add functions to (un)register GISC with GISA
In reply to Michael MuellerAdd the IAM (Interruption Alert Mask) to the architecture specific kvm struct. This mask in the GISA is used to define for which ISC a GIB alert can be issued.
The functions kvm_s390_gisc_register() and kvm_s390_gisc_unregister() are used to (un)register a GISC (guest ISC) with a virtual machine and its GISA.
Upon successful completion, kvm_s390_gisc_register() returns the ISC to be used for GIB alert interruptions. A negative return code indicates an error during registration.
Theses functions will be used by other adapter types like AP and PCI to request pass-through interruption support.
--- arch/s390/include/asm/kvm_host.h | 9 ++++++ arch/s390/kvm/interrupt.c | 66 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+)
arch/s390/include/asm/kvm_host.h +9 -0
@@ -782,6 +782,9 @@ struct kvm_s390_gisa {u8 reserved03[11 ];u32 airq_count; } g1; +struct { +u64 word[4 ]; + } u64; }; };
@@ -849,6 +852,9 @@ struct kvm_arch{DECLARE_BITMAP (cpu_feat,KVM_S390_VM_CPU_FEAT_NR_BITS );struct kvm_s390_gisa *gisa;int gib_in_use; +u8 iam; +u32 iam_ref_count[MAX_ISC +1 ]; +spinlock_t iam_ref_lock; };#define KVM_HVA_ERR_BAD (-1UL)
@@ -882,6 +888,9 @@ void kvm_arch_crypto_set_masks(struct kvm *kvm, unsigned long *apm,extern int sie64a (struct kvm_s390_sie_block *,u64 *);extern char sie_exit; +extern int kvm_s390_gisc_register (struct kvm *kvm,u32 gisc); +extern int kvm_s390_gisc_unregister (struct kvm *kvm,u32 gisc); +static inline void kvm_arch_hardware_disable (void ) {}static inline void kvm_arch_check_processor_compat (void *rtn) {}static inline void kvm_arch_sync_events (struct kvm *kvm) {}
arch/s390/kvm/interrupt.c +66 -0
84 lines not on this page · raw message
Signature · 1 line
2.13.4
Re: [PATCH v5 10/15] KVM: s390: add functions to (un)register GISC with GISA
Quoted 21 lines · Michael Mueller
Add the IAM (Interruption Alert Mask) to the architecture specific kvm struct. This mask in the GISA is used to define for which ISC a GIB alert can be issued.
The functions
kvm_s390_gisc_register()andkvm_s390_gisc_unregister()are used to (un)register a GISC (guest ISC) with a virtual machine and its GISA.Upon successful completion,
kvm_s390_gisc_register()returns the ISC to be used for GIB alert interruptions. A negative return code indicates an error during registration.Theses functions will be used by other adapter types like AP and PCI to request pass-through interruption support.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/include/asm/kvm_host.h | 9 ++++++ arch/s390/kvm/interrupt.c | 66 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+)
...
arch/s390/kvm/interrupt.c static inline u8 int_word_to_isc(u32 int_word) · line 232
Quoted 7 lines
arch/s390/kvm/interrupt.c +1
-0@@ -229,6 +229,25 @@ static inline u8 int_word_to_isc(u32 int_word)*/ #define IPM_BIT_OFFSET (offsetof(struct kvm_s390_gisa, ipm) * BITS_PER_BYTE) +static inline int set_iam (struct kvm_s390_gisa *gisa,u8 iam)
I will rename this one here as well to gisa_set_iam()
Quoted 18 lines
arch/s390/kvm/interrupt.c +18
-0+{ +u64 word0, _word0; + +do { + word0 =READ_ONCE (gisa->u64.word[0 ]); +/* If the GISA is in the alert list, do nothing. */ +if ((u64 )gisa != word0 >>32 ) +return -EBUSY ; +/* +* Try to set the IAM or loop, if the IPM has changed +* or the GISA has been inserted into the alert list. +*/ + _word0 = (word0 & ~0xffUL ) | iam; + }while (cmpxchg (&gisa->u64.word[0 ], word0, _word0) != _word0); + +return 0 ; +} +
Re: [PATCH v5 10/15] KVM: s390: add functions to (un)register GISC with GISA
In reply to Michael MuellerQuoted 62 lines · Michael Mueller
Add the IAM (Interruption Alert Mask) to the architecture specific kvm struct. This mask in the GISA is used to define for which ISC a GIB alert can be issued.
The functions
kvm_s390_gisc_register()andkvm_s390_gisc_unregister()are used to (un)register a GISC (guest ISC) with a virtual machine and its GISA.Upon successful completion,
kvm_s390_gisc_register()returns the ISC to be used for GIB alert interruptions. A negative return code indicates an error during registration.Theses functions will be used by other adapter types like AP and PCI to request pass-through interruption support.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/include/asm/kvm_host.h | 9 ++++++ arch/s390/kvm/interrupt.c | 66 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+)arch/s390/include/asm/kvm_host.h +9
-0@@ -782,6 +782,9 @@ struct kvm_s390_gisa {u8 reserved03[11 ];u32 airq_count; } g1; +struct { +u64 word[4 ]; + } u64; }; };@@ -849,6 +852,9 @@ struct kvm_arch{DECLARE_BITMAP (cpu_feat,KVM_S390_VM_CPU_FEAT_NR_BITS );struct kvm_s390_gisa *gisa;int gib_in_use; +u8 iam; +u32 iam_ref_count[MAX_ISC +1 ]; +spinlock_t iam_ref_lock; };#define KVM_HVA_ERR_BAD (-1UL)@@ -882,6 +888,9 @@ void kvm_arch_crypto_set_masks(struct kvm *kvm, unsigned long *apm,extern int sie64a (struct kvm_s390_sie_block *,u64 *);extern char sie_exit; +extern int kvm_s390_gisc_register (struct kvm *kvm,u32 gisc); +extern int kvm_s390_gisc_unregister (struct kvm *kvm,u32 gisc); +static inline void kvm_arch_hardware_disable (void ) {}static inline void kvm_arch_check_processor_compat (void *rtn) {}static inline void kvm_arch_sync_events (struct kvm *kvm) {}arch/s390/kvm/interrupt.c +0
-0@@ -229,6 +229,25 @@ static inline u8 int_word_to_isc(u32 int_word)*/ #define IPM_BIT_OFFSET (offsetof(struct kvm_s390_gisa, ipm) * BITS_PER_BYTE)
arch/s390/kvm/interrupt.c static inline u8 int_word_to_isc(u32 int_word) · line 240
Quoted 10 lines
arch/s390/kvm/interrupt.c +9
-0+static inline int set_iam (struct kvm_s390_gisa *gisa,u8 iam) +{ +u64 word0, _word0; + + do { + word0 = READ_ONCE(gisa->u64.word[0]); + /* If the GISA is in the alert list, do nothing. */ + if ((u64)gisa != word0 >> 32) + return -EBUSY;
In kvm_s390_gisa_clear() you do:
+ nullify_gisa(kvm->arch.gisa);
+ set_iam(kvm->arch.gisa, kvm->arch.iam);
IIUC this call to set_iam() will always fail.
Quoted 28 lines
arch/s390/kvm/interrupt.c +14
-0+/* +* Try to set the IAM or loop, if the IPM has changed +* or the GISA has been inserted into the alert list. +*/ + _word0 = (word0 & ~0xffUL ) | iam; + } while (cmpxchg(&gisa->u64.word[0], word0, _word0) != _word0); > + + return 0; +} +static inline void set_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc) {set_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa);@@ -2917,6 +2936,8 @@ void kvm_s390_gisa_init(struct kvm *kvm)if (!css_general_characteristics.aiv)return ; kvm->arch.gisa = &kvm->arch.sie_page2->gisa; + kvm->arch.iam =0 ; +spin_lock_init (&kvm->arch.iam_ref_lock);nullify_gisa (kvm->arch.gisa); kvm->arch.gib_in_use = !!gib;VM_EVENT (kvm,3 ,"gisa 0x%pK initialized" , kvm->arch.gisa);@@ -2929,6 +2950,51 @@ void kvm_s390_gisa_destroy(struct kvm *kvm) kvm->arch.gisa =NULL ; } +int kvm_s390_gisc_register (struct kvm *kvm,u32 gisc) +{ +if (!kvm->arch.gib_in_use)
arch/s390/kvm/interrupt.c void kvm_s390_gisa_destroy(struct kvm *kvm) · line 2965
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+return -ENODEV ; +if (gisc >MAX_ISC ) +return -ERANGE ; + +spin_lock (&kvm->arch.iam_ref_lock); +if (kvm->arch.iam_ref_count[gisc] ==0 ) + kvm->arch.iam |=0x80 >> gisc; + kvm->arch.iam_ref_count[gisc]++; +if (kvm->arch.iam_ref_count[gisc] ==1 ) +set_iam (kvm->arch.gisa, kvm->arch.iam);
testing the set_iam return value?
Even it should be fine if the caller works correctly, this is done before GISA is ever used.
Quoted 36 lines
arch/s390/kvm/interrupt.c +32
-0+spin_unlock (&kvm->arch.iam_ref_lock); + +return gib->nisc; +} +EXPORT_SYMBOL_GPL (kvm_s390_gisc_register); + +int kvm_s390_gisc_unregister (struct kvm *kvm,u32 gisc) +{ +int rc =0 ; + +if (!kvm->arch.gib_in_use) +return -ENODEV ; +if (gisc >MAX_ISC ) +return -ERANGE ; + +spin_lock (&kvm->arch.iam_ref_lock); +if (kvm->arch.iam_ref_count[gisc] ==0 ) { + rc = -EINVAL ; + goto out; + } + kvm->arch.iam_ref_count[gisc]--; +if (kvm->arch.iam_ref_count[gisc] ==0 ) { + kvm->arch.iam &= ~(0x80 >> gisc); +set_iam (kvm->arch.gisa, kvm->arch.iam); + } +out: +spin_unlock (&kvm->arch.iam_ref_lock); + +return rc; +} +EXPORT_SYMBOL_GPL (kvm_s390_gisc_unregister); +void kvm_s390_gib_destroy (void ) {if (!gib)
Signature · 2 lines
Pierre Morel Linux/KVM/QEMU in Böblingen - Germany
Re: [PATCH v5 10/15] KVM: s390: add functions to (un)register GISC with GISA
Quoted 74 lines · Pierre Morel
Quoted 60 lines · Michael Mueller
Add the IAM (Interruption Alert Mask) to the architecture specific kvm struct. This mask in the GISA is used to define for which ISC a GIB alert can be issued.
The functions
kvm_s390_gisc_register()andkvm_s390_gisc_unregister()are used to (un)register a GISC (guest ISC) with a virtual machine and its GISA.Upon successful completion,
kvm_s390_gisc_register()returns the ISC to be used for GIB alert interruptions. A negative return code indicates an error during registration.Theses functions will be used by other adapter types like AP and PCI to request pass-through interruption support.
Signed-off-by Michael Mueller <[email protected]>---
arch/s390/include/asm/kvm_host.h| 9 ++++++
arch/s390/kvm/interrupt.c | 66
++++++++++++++++++++++++++++++++++++++++2 files changed, 75 insertions(+)arch/s390/include/asm/kvm_host.h +9
-0@@ -782,6 +782,9 @@ struct kvm_s390_gisa { u8 reserved03[11]; u32 airq_count; } g1; + struct { + u64 word[4]; + } u64; }; };@@ -849,6 +852,9 @@ struct kvm_arch{ DECLARE_BITMAP(cpu_feat, KVM_S390_VM_CPU_FEAT_NR_BITS); struct kvm_s390_gisa *gisa; int gib_in_use; + u8 iam; + u32 iam_ref_count[MAX_ISC + 1]; + spinlock_t iam_ref_lock; }; #define KVM_HVA_ERR_BAD (-1UL)@@ -882,6 +888,9 @@ void kvm_arch_crypto_set_masks(struct kvm *kvm,unsigned long *apm, extern int sie64a(struct kvm_s390_sie_block *, u64 *); extern char sie_exit; +extern int kvm_s390_gisc_register (struct kvm *kvm,u32 gisc); +extern int kvm_s390_gisc_unregister (struct kvm *kvm,u32 gisc); + static inline void kvm_arch_hardware_disable(void) {} static inline void kvm_arch_check_processor_compat(void *rtn) {} static inline void kvm_arch_sync_events(struct kvm *kvm) {}arch/s390/kvm/interrupt.c +0
-0@@ -229,6 +229,25 @@ static inline u8 int_word_to_isc(u32 int_word)*/ arch/s390/kvm/interrupt.c static inline u8 int_word_to_isc(u32 int_word) · line 239
Quoted 10 lines
arch/s390/kvm/interrupt.c +9
-0#define IPM_BIT_OFFSET (offsetof(struct kvm_s390_gisa, ipm) * BITS_PER_BYTE) +static inline int set_iam (struct kvm_s390_gisa *gisa,u8 iam) +{ + u64 word0, _word0; + + do { + word0 = READ_ONCE(gisa->u64.word[0]); + /* If the GISA is in the alert list, do nothing. */ + if ((u64)gisa != word0 >> 32) + return -EBUSY;In
kvm_s390_gisa_clear()you do:
Quoted 4 lines
+
nullify_gisa(kvm->arch.gisa);
+set_iam(kvm->arch.gisa, kvm->arch.iam);IIUC this call to
set_iam()will always fail.
Hum, sorry, I oversaw a patch which corrected this problem. Sorry for the noise.
(I should have seen it since I copied the right lines!)
Regards,
Pierre
Signature · 2 lines
Pierre Morel Linux/KVM/QEMU in Böblingen - Germany
Re: [PATCH v5 10/15] KVM: s390: add functions to (un)register GISC with GISA
In reply to Pierre MorelQuoted 32 lines · On Wed, 2 Jan 2019 18:29:00 +0100 Pierre Morel <[email protected]> wrote:
Quoted 31 lines · Michael Mueller
Add the IAM (Interruption Alert Mask) to the architecture specific kvm struct. This mask in the GISA is used to define for which ISC a GIB alert can be issued.
The functions
kvm_s390_gisc_register()andkvm_s390_gisc_unregister()are used to (un)register a GISC (guest ISC) with a virtual machine and its GISA.Upon successful completion,
kvm_s390_gisc_register()returns the ISC to be used for GIB alert interruptions. A negative return code indicates an error during registration.Theses functions will be used by other adapter types like AP and PCI to request pass-through interruption support.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/include/asm/kvm_host.h | 9 ++++++ arch/s390/kvm/interrupt.c | 66 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+)arch/s390/kvm/interrupt.c +9
-0+int kvm_s390_gisc_register (struct kvm *kvm,u32 gisc) +{ +if (!kvm->arch.gib_in_use) +return -ENODEV ; +if (gisc >MAX_ISC ) +return -ERANGE ; + +spin_lock (&kvm->arch.iam_ref_lock); +if (kvm->arch.iam_ref_count[gisc] ==0 )
Quoted 8 lines
arch/s390/kvm/interrupt.c void kvm_s390_gisa_destroy(struct kvm *kvm) · line 2965
Quoted 4 lines
arch/s390/kvm/interrupt.c +4
-0+ kvm->arch.iam |=0x80 >> gisc; + kvm->arch.iam_ref_count[gisc]++; +if (kvm->arch.iam_ref_count[gisc] ==1 ) +set_iam (kvm->arch.gisa, kvm->arch.iam);testing the
set_iamreturn value?
Even it should be fine if the caller works correctly, this is done before GISA is ever used.
My feeling is that checking the return code is a good idea, even if it Should Never Fail(tm).
Quoted 17 lines
Quoted 17 lines
arch/s390/kvm/interrupt.c +16
-0+spin_unlock (&kvm->arch.iam_ref_lock); + +return gib->nisc; +} +EXPORT_SYMBOL_GPL (kvm_s390_gisc_register); + +int kvm_s390_gisc_unregister (struct kvm *kvm,u32 gisc) +{ +int rc =0 ; + +if (!kvm->arch.gib_in_use) +return -ENODEV ; +if (gisc >MAX_ISC ) +return -ERANGE ; + +spin_lock (&kvm->arch.iam_ref_lock);
Quoted 8 lines
arch/s390/kvm/interrupt.c void kvm_s390_gisa_destroy(struct kvm *kvm) · line 2989
Quoted 8 lines
arch/s390/kvm/interrupt.c +8
-0+if (kvm->arch.iam_ref_count[gisc] ==0 ) { + rc = -EINVAL ; + goto out; + } + kvm->arch.iam_ref_count[gisc]--; +if (kvm->arch.iam_ref_count[gisc] ==0 ) { + kvm->arch.iam &= ~(0x80 >> gisc); +set_iam (kvm->arch.gisa, kvm->arch.iam);
Any chance of this function failing here? If yes, would there be any implications?
Quoted 14 lines
Quoted 12 lines
arch/s390/kvm/interrupt.c +8
-0+ } +out: +spin_unlock (&kvm->arch.iam_ref_lock); + +return rc; +} +EXPORT_SYMBOL_GPL (kvm_s390_gisc_unregister); +void kvm_s390_gib_destroy (void ) {if (!gib)
Re: [PATCH v5 10/15] KVM: s390: add functions to (un)register GISC with GISA
Quoted 35 lines · Cornelia Huck
Quoted 32 lines · On Wed, 2 Jan 2019 18:29:00 +0100 Pierre Morel <[email protected]> wrote:
Quoted 31 lines · Michael Mueller
Add the IAM (Interruption Alert Mask) to the architecture specific kvm struct. This mask in the GISA is used to define for which ISC a GIB alert can be issued.
The functions
kvm_s390_gisc_register()andkvm_s390_gisc_unregister()are used to (un)register a GISC (guest ISC) with a virtual machine and its GISA.Upon successful completion,
kvm_s390_gisc_register()returns the ISC to be used for GIB alert interruptions. A negative return code indicates an error during registration.Theses functions will be used by other adapter types like AP and PCI to request pass-through interruption support.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/include/asm/kvm_host.h | 9 ++++++ arch/s390/kvm/interrupt.c | 66 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+)arch/s390/kvm/interrupt.c +9
-0+int kvm_s390_gisc_register (struct kvm *kvm,u32 gisc) +{ +if (!kvm->arch.gib_in_use) +return -ENODEV ; +if (gisc >MAX_ISC ) +return -ERANGE ; + +spin_lock (&kvm->arch.iam_ref_lock); +if (kvm->arch.iam_ref_count[gisc] ==0 )
Quoted 8 lines
Quoted 8 lines
arch/s390/kvm/interrupt.c void kvm_s390_gisa_destroy(struct kvm *kvm) · line 2965
Quoted 4 lines
arch/s390/kvm/interrupt.c +4
-0+ kvm->arch.iam |=0x80 >> gisc; + kvm->arch.iam_ref_count[gisc]++; +if (kvm->arch.iam_ref_count[gisc] ==1 ) +set_iam (kvm->arch.gisa, kvm->arch.iam);testing the
set_iamreturn value?
Even it should be fine if the caller works correctly, this is done before GISA is ever used.
There is a rc but a check here is not required.
There are three cases:
a) This is the first ISC that gets registered, then the GISA is not in use and IAM is set in the GISA.
b) A second ISC gets registered and the GISA is *not* in the alert list. Then the IAM is set here as well.
c) A second ISC gets registered and the GISA is in the alert list. Then the IAM is intentionally not set here by set_iam(). It will be restored by get_ipm() with the new IAM value by the gib alert processing code.
Quoted 24 lines
My feeling is that checking the return code is a good idea, even if it Should Never Fail(tm).
Quoted 20 lines
Quoted 20 lines
arch/s390/kvm/interrupt.c +19
-0+spin_unlock (&kvm->arch.iam_ref_lock); + +return gib->nisc; +} +EXPORT_SYMBOL_GPL (kvm_s390_gisc_register); + +int kvm_s390_gisc_unregister (struct kvm *kvm,u32 gisc) +{ +int rc =0 ; + +if (!kvm->arch.gib_in_use) +return -ENODEV ; +if (gisc >MAX_ISC ) +return -ERANGE ; + +spin_lock (&kvm->arch.iam_ref_lock); +if (kvm->arch.iam_ref_count[gisc] ==0 ) { + rc = -EINVAL ; + goto out;
Quoted 8 lines
Quoted 5 lines
arch/s390/kvm/interrupt.c void kvm_s390_gisa_destroy(struct kvm *kvm) · line 2989
Quoted 5 lines
arch/s390/kvm/interrupt.c +5
-0+ } + kvm->arch.iam_ref_count[gisc]--; +if (kvm->arch.iam_ref_count[gisc] ==0 ) { + kvm->arch.iam &= ~(0x80 >> gisc); +set_iam (kvm->arch.gisa, kvm->arch.iam);Any chance of this function failing here? If yes, would there be any implications?
It is the same here.
Quoted 16 lines
Quoted 15 lines
Quoted 13 lines
arch/s390/kvm/interrupt.c +8
-0+ } +out: +spin_unlock (&kvm->arch.iam_ref_lock); + +return rc; +} +EXPORT_SYMBOL_GPL (kvm_s390_gisc_unregister); +void kvm_s390_gib_destroy (void ) {if (!gib)
Re: [PATCH v5 10/15] KVM: s390: add functions to (un)register GISC with GISA
Quoted 94 lines · On Mon, 7 Jan 2019 18:38:02 +0100 Michael Mueller <[email protected]> wrote:
Quoted 35 lines · Cornelia Huck
Quoted 32 lines · On Wed, 2 Jan 2019 18:29:00 +0100 Pierre Morel <[email protected]> wrote:
Quoted 31 lines · Michael Mueller
Add the IAM (Interruption Alert Mask) to the architecture specific kvm struct. This mask in the GISA is used to define for which ISC a GIB alert can be issued.
The functions
kvm_s390_gisc_register()andkvm_s390_gisc_unregister()are used to (un)register a GISC (guest ISC) with a virtual machine and its GISA.Upon successful completion,
kvm_s390_gisc_register()returns the ISC to be used for GIB alert interruptions. A negative return code indicates an error during registration.Theses functions will be used by other adapter types like AP and PCI to request pass-through interruption support.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/include/asm/kvm_host.h | 9 ++++++ arch/s390/kvm/interrupt.c | 66 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+)arch/s390/kvm/interrupt.c +9
-0+int kvm_s390_gisc_register (struct kvm *kvm,u32 gisc) +{ +if (!kvm->arch.gib_in_use) +return -ENODEV ; +if (gisc >MAX_ISC ) +return -ERANGE ; + +spin_lock (&kvm->arch.iam_ref_lock); +if (kvm->arch.iam_ref_count[gisc] ==0 )Quoted 8 lines
Quoted 8 lines
arch/s390/kvm/interrupt.c void kvm_s390_gisa_destroy(struct kvm *kvm) · line 2965
Quoted 4 lines
arch/s390/kvm/interrupt.c +4
-0+ kvm->arch.iam |=0x80 >> gisc; + kvm->arch.iam_ref_count[gisc]++; +if (kvm->arch.iam_ref_count[gisc] ==1 ) +set_iam (kvm->arch.gisa, kvm->arch.iam);testing the
set_iamreturn value?
Even it should be fine if the caller works correctly, this is done before GISA is ever used.There is a rc but a check here is not required.
There are three cases:
a) This is the first ISC that gets registered, then the GISA is not in use and IAM is set in the GISA.
b) A second ISC gets registered and the GISA is *not* in the alert list. Then the IAM is set here as well.
c) A second ISC gets registered and the GISA is in the alert list. Then the IAM is intentionally not set here by
set_iam(). It will be restored byget_ipm()with the new IAM value by the gib alert processing code.Quoted 24 lines
My feeling is that checking the return code is a good idea, even if it Should Never Fail(tm).
Quoted 20 lines
Quoted 20 lines
arch/s390/kvm/interrupt.c +19
-0+spin_unlock (&kvm->arch.iam_ref_lock); + +return gib->nisc; +} +EXPORT_SYMBOL_GPL (kvm_s390_gisc_register); + +int kvm_s390_gisc_unregister (struct kvm *kvm,u32 gisc) +{ +int rc =0 ; + +if (!kvm->arch.gib_in_use) +return -ENODEV ; +if (gisc >MAX_ISC ) +return -ERANGE ; + +spin_lock (&kvm->arch.iam_ref_lock); +if (kvm->arch.iam_ref_count[gisc] ==0 ) { + rc = -EINVAL ; + goto out;Quoted 8 lines
Quoted 5 lines
arch/s390/kvm/interrupt.c void kvm_s390_gisa_destroy(struct kvm *kvm) · line 2989
Quoted 5 lines
arch/s390/kvm/interrupt.c +5
-0+ } + kvm->arch.iam_ref_count[gisc]--; +if (kvm->arch.iam_ref_count[gisc] ==0 ) { + kvm->arch.iam &= ~(0x80 >> gisc); +set_iam (kvm->arch.gisa, kvm->arch.iam);Any chance of this function failing here? If yes, would there be any implications?
Quoted 1 line
It is the same here.
I'm not sure that I follow: This is the reverse operation (unregistering the gisc). Can we rely on get_ipm() to do any fixup later? Is that a problem for the caller?
Apologies if I sound confused (well, that's because I probably am); this is hard to review without access to the hardware specification.
Quoted 18 lines
Quoted 17 lines
Quoted 16 lines
Quoted 14 lines
arch/s390/kvm/interrupt.c +8
-0+ } +out: +spin_unlock (&kvm->arch.iam_ref_lock); + +return rc; +} +EXPORT_SYMBOL_GPL (kvm_s390_gisc_unregister); +void kvm_s390_gib_destroy (void ) {if (!gib)
Re: [PATCH v5 10/15] KVM: s390: add functions to (un)register GISC with GISA
Quoted 97 lines · Cornelia Huck
Quoted 94 lines · On Mon, 7 Jan 2019 18:38:02 +0100 Michael Mueller <[email protected]> wrote:
Quoted 35 lines · Cornelia Huck
Quoted 32 lines · On Wed, 2 Jan 2019 18:29:00 +0100 Pierre Morel <[email protected]> wrote:
Quoted 31 lines · Michael Mueller
Add the IAM (Interruption Alert Mask) to the architecture specific kvm struct. This mask in the GISA is used to define for which ISC a GIB alert can be issued.
The functions
kvm_s390_gisc_register()andkvm_s390_gisc_unregister()are used to (un)register a GISC (guest ISC) with a virtual machine and its GISA.Upon successful completion,
kvm_s390_gisc_register()returns the ISC to be used for GIB alert interruptions. A negative return code indicates an error during registration.Theses functions will be used by other adapter types like AP and PCI to request pass-through interruption support.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/include/asm/kvm_host.h | 9 ++++++ arch/s390/kvm/interrupt.c | 66 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+)arch/s390/kvm/interrupt.c +9
-0+int kvm_s390_gisc_register (struct kvm *kvm,u32 gisc) +{ +if (!kvm->arch.gib_in_use) +return -ENODEV ; +if (gisc >MAX_ISC ) +return -ERANGE ; + +spin_lock (&kvm->arch.iam_ref_lock); +if (kvm->arch.iam_ref_count[gisc] ==0 )Quoted 8 lines
Quoted 8 lines
arch/s390/kvm/interrupt.c void kvm_s390_gisa_destroy(struct kvm *kvm) · line 2965
Quoted 4 lines
arch/s390/kvm/interrupt.c +4
-0+ kvm->arch.iam |=0x80 >> gisc; + kvm->arch.iam_ref_count[gisc]++; +if (kvm->arch.iam_ref_count[gisc] ==1 ) +set_iam (kvm->arch.gisa, kvm->arch.iam);testing the
set_iamreturn value?
Even it should be fine if the caller works correctly, this is done before GISA is ever used.There is a rc but a check here is not required.
There are three cases:
a) This is the first ISC that gets registered, then the GISA is not in use and IAM is set in the GISA.
b) A second ISC gets registered and the GISA is *not* in the alert list. Then the IAM is set here as well.
c) A second ISC gets registered and the GISA is in the alert list. Then the IAM is intentionally not set here by
set_iam(). It will be restored byget_ipm()with the new IAM value by the gib alert processing code.Quoted 32 lines
My feeling is that checking the return code is a good idea, even if it Should Never Fail(tm).
Quoted 17 lines
Quoted 17 lines
arch/s390/kvm/interrupt.c +16
-0+spin_unlock (&kvm->arch.iam_ref_lock); + +return gib->nisc; +} +EXPORT_SYMBOL_GPL (kvm_s390_gisc_register); + +int kvm_s390_gisc_unregister (struct kvm *kvm,u32 gisc) +{ +int rc =0 ; + +if (!kvm->arch.gib_in_use) +return -ENODEV ; +if (gisc >MAX_ISC ) +return -ERANGE ; + +spin_lock (&kvm->arch.iam_ref_lock);Quoted 8 lines
arch/s390/kvm/interrupt.c void kvm_s390_gisa_destroy(struct kvm *kvm) · line 2989
Quoted 8 lines
arch/s390/kvm/interrupt.c +8
-0+if (kvm->arch.iam_ref_count[gisc] ==0 ) { + rc = -EINVAL ; + goto out; + } + kvm->arch.iam_ref_count[gisc]--; +if (kvm->arch.iam_ref_count[gisc] ==0 ) { + kvm->arch.iam &= ~(0x80 >> gisc); +set_iam (kvm->arch.gisa, kvm->arch.iam);Any chance of this function failing here? If yes, would there be any implications?
Quoted 8 lines
Quoted 1 line
It is the same here.
I'm not sure that I follow: This is the reverse operation (unregistering the gisc). Can we rely on
get_ipm()to do any fixup later? Is that a problem for the caller?Apologies if I sound confused (well, that's because I probably am); this is hard to review without access to the hardware specification.
I think nothing will happen because the AP CLR IRQ call (Pierre?) has already taken offline the last AP device.
Quoted 20 lines
Quoted 19 lines
Quoted 18 lines
Quoted 17 lines
Quoted 15 lines
arch/s390/kvm/interrupt.c +8
-0+ } +out: +spin_unlock (&kvm->arch.iam_ref_lock); + +return rc; +} +EXPORT_SYMBOL_GPL (kvm_s390_gisc_unregister); +void kvm_s390_gib_destroy (void ) {if (!gib)
Re: [PATCH v5 10/15] KVM: s390: add functions to (un)register GISC with GISA
Quoted 104 lines · On Tue, 8 Jan 2019 14:07:06 +0100 Michael Mueller <[email protected]> wrote:
Quoted 103 lines · Cornelia Huck
Quoted 94 lines · On Mon, 7 Jan 2019 18:38:02 +0100 Michael Mueller <[email protected]> wrote:
Quoted 35 lines · Cornelia Huck
Quoted 32 lines · On Wed, 2 Jan 2019 18:29:00 +0100 Pierre Morel <[email protected]> wrote:
Quoted 31 lines · Michael Mueller
Add the IAM (Interruption Alert Mask) to the architecture specific kvm struct. This mask in the GISA is used to define for which ISC a GIB alert can be issued.
The functions
kvm_s390_gisc_register()andkvm_s390_gisc_unregister()are used to (un)register a GISC (guest ISC) with a virtual machine and its GISA.Upon successful completion,
kvm_s390_gisc_register()returns the ISC to be used for GIB alert interruptions. A negative return code indicates an error during registration.Theses functions will be used by other adapter types like AP and PCI to request pass-through interruption support.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/include/asm/kvm_host.h | 9 ++++++ arch/s390/kvm/interrupt.c | 66 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 75 insertions(+)arch/s390/kvm/interrupt.c +9
-0+int kvm_s390_gisc_register (struct kvm *kvm,u32 gisc) +{ +if (!kvm->arch.gib_in_use) +return -ENODEV ; +if (gisc >MAX_ISC ) +return -ERANGE ; + +spin_lock (&kvm->arch.iam_ref_lock); +if (kvm->arch.iam_ref_count[gisc] ==0 )Quoted 8 lines
Quoted 8 lines
arch/s390/kvm/interrupt.c void kvm_s390_gisa_destroy(struct kvm *kvm) · line 2965
Quoted 4 lines
arch/s390/kvm/interrupt.c +4
-0+ kvm->arch.iam |=0x80 >> gisc; + kvm->arch.iam_ref_count[gisc]++; +if (kvm->arch.iam_ref_count[gisc] ==1 ) +set_iam (kvm->arch.gisa, kvm->arch.iam);testing the
set_iamreturn value?
Even it should be fine if the caller works correctly, this is done before GISA is ever used.There is a rc but a check here is not required.
There are three cases:
a) This is the first ISC that gets registered, then the GISA is not in use and IAM is set in the GISA.
b) A second ISC gets registered and the GISA is *not* in the alert list. Then the IAM is set here as well.
c) A second ISC gets registered and the GISA is in the alert list. Then the IAM is intentionally not set here by
set_iam(). It will be restored byget_ipm()with the new IAM value by the gib alert processing code.Quoted 24 lines
My feeling is that checking the return code is a good idea, even if it Should Never Fail(tm).
Quoted 20 lines
Quoted 20 lines
arch/s390/kvm/interrupt.c +19
-0+spin_unlock (&kvm->arch.iam_ref_lock); + +return gib->nisc; +} +EXPORT_SYMBOL_GPL (kvm_s390_gisc_register); + +int kvm_s390_gisc_unregister (struct kvm *kvm,u32 gisc) +{ +int rc =0 ; + +if (!kvm->arch.gib_in_use) +return -ENODEV ; +if (gisc >MAX_ISC ) +return -ERANGE ; + +spin_lock (&kvm->arch.iam_ref_lock); +if (kvm->arch.iam_ref_count[gisc] ==0 ) { + rc = -EINVAL ; + goto out;Quoted 8 lines
Quoted 5 lines
arch/s390/kvm/interrupt.c void kvm_s390_gisa_destroy(struct kvm *kvm) · line 2989
Quoted 5 lines
arch/s390/kvm/interrupt.c +5
-0+ } + kvm->arch.iam_ref_count[gisc]--; +if (kvm->arch.iam_ref_count[gisc] ==0 ) { + kvm->arch.iam &= ~(0x80 >> gisc); +set_iam (kvm->arch.gisa, kvm->arch.iam);Any chance of this function failing here? If yes, would there be any implications?
Quoted 1 line
It is the same here.
I'm not sure that I follow: This is the reverse operation (unregistering the gisc). Can we rely on
get_ipm()to do any fixup later? Is that a problem for the caller?
Quoted 5 lines
Quoted 2 lines
Apologies if I sound confused (well, that's because I probably am); this is hard to review without access to the hardware specification.
I think nothing will happen because the AP CLR IRQ call (Pierre?) has already taken offline the last AP device.
But doesn't that rely on behaviour by the caller?
If the caller does weird and broken stuff, this function should return an error, shouldn't it?
Quoted 23 lines
Quoted 22 lines
Quoted 21 lines
Quoted 20 lines
Quoted 19 lines
Quoted 17 lines
arch/s390/kvm/interrupt.c +8
-0+ } +out: +spin_unlock (&kvm->arch.iam_ref_lock); + +return rc; +} +EXPORT_SYMBOL_GPL (kvm_s390_gisc_unregister); +void kvm_s390_gib_destroy (void ) {if (!gib)
Re: [PATCH v5 10/15] KVM: s390: add functions to (un)register GISC with GISA
In reply to Cornelia HuckQuoted 29 lines · On Tue, 8 Jan 2019 11:34:44 +0100 Cornelia Huck <[email protected]> wrote:
Quoted 29 lines
Quoted 28 lines
Quoted 17 lines
Quoted 17 lines
arch/s390/kvm/interrupt.c +16
-0+spin_unlock (&kvm->arch.iam_ref_lock); + +return gib->nisc; +} +EXPORT_SYMBOL_GPL (kvm_s390_gisc_register); + +int kvm_s390_gisc_unregister (struct kvm *kvm,u32 gisc) +{ +int rc =0 ; + +if (!kvm->arch.gib_in_use) +return -ENODEV ; +if (gisc >MAX_ISC ) +return -ERANGE ; + +spin_lock (&kvm->arch.iam_ref_lock);Quoted 8 lines
arch/s390/kvm/interrupt.c void kvm_s390_gisa_destroy(struct kvm *kvm) · line 2989
Quoted 8 lines
arch/s390/kvm/interrupt.c +8
-0+if (kvm->arch.iam_ref_count[gisc] ==0 ) { + rc = -EINVAL ; + goto out; + } + kvm->arch.iam_ref_count[gisc]--; +if (kvm->arch.iam_ref_count[gisc] ==0 ) { + kvm->arch.iam &= ~(0x80 >> gisc); +set_iam (kvm->arch.gisa, kvm->arch.iam);Any chance of this function failing here? If yes, would there be any implications?
Quoted 5 lines
Quoted 1 line
It is the same here.
I'm not sure that I follow: This is the reverse operation (unregistering the gisc). Can we rely on
get_ipm()to do any fixup later? Is that a problem for the caller?
IMHO gib alerts are all about not loosing initiative. I.e. avoiding substantially delayed delivery of interrupts due to vCPUs in wait state.
Thus we must avoid letting go before setting up stuff (gisa.iam under consideration of gisa ipm, gisa.next_alert, and gib.alert_list_origin) so that we can react on the next interrupt (if necessary).
On the other hand, getting more gisa alerts than necessary is not fatal -- better avoided if not too much hassle of course.
Bottom line is, while it may be critical that the IAM changes implied by register take place immediately, unregister is a different story.
Regards,
Halil
Quoted 3 lines
Apologies if I sound confused (well, that's because I probably am); this is hard to review without access to the hardware specification.
Re: [PATCH v5 10/15] KVM: s390: add functions to (un)register GISC with GISA
Quoted 42 lines · On Tue, 8 Jan 2019 14:36:13 +0100 Halil Pasic <[email protected]> wrote:
Quoted 29 lines · On Tue, 8 Jan 2019 11:34:44 +0100 Cornelia Huck <[email protected]> wrote:
Quoted 29 lines
Quoted 28 lines
Quoted 17 lines
Quoted 17 lines
arch/s390/kvm/interrupt.c +16
-0+spin_unlock (&kvm->arch.iam_ref_lock); + +return gib->nisc; +} +EXPORT_SYMBOL_GPL (kvm_s390_gisc_register); + +int kvm_s390_gisc_unregister (struct kvm *kvm,u32 gisc) +{ +int rc =0 ; + +if (!kvm->arch.gib_in_use) +return -ENODEV ; +if (gisc >MAX_ISC ) +return -ERANGE ; + +spin_lock (&kvm->arch.iam_ref_lock);Quoted 8 lines
arch/s390/kvm/interrupt.c void kvm_s390_gisa_destroy(struct kvm *kvm) · line 2989
Quoted 8 lines
arch/s390/kvm/interrupt.c +8
-0+if (kvm->arch.iam_ref_count[gisc] ==0 ) { + rc = -EINVAL ; + goto out; + } + kvm->arch.iam_ref_count[gisc]--; +if (kvm->arch.iam_ref_count[gisc] ==0 ) { + kvm->arch.iam &= ~(0x80 >> gisc); +set_iam (kvm->arch.gisa, kvm->arch.iam);Any chance of this function failing here? If yes, would there be any implications?
Quoted 5 lines
Quoted 1 line
It is the same here.
I'm not sure that I follow: This is the reverse operation (unregistering the gisc). Can we rely on
get_ipm()to do any fixup later? Is that a problem for the caller?IMHO gib alerts are all about not loosing initiative. I.e. avoiding substantially delayed delivery of interrupts due to vCPUs in wait state.
Quoted 6 lines
Thus we must avoid letting go before setting up stuff (gisa.iam under consideration of gisa ipm, gisa.next_alert, and gib.alert_list_origin) so that we can react on the next interrupt (if necessary).
On the other hand, getting more gisa alerts than necessary is not fatal -- better avoided if not too much hassle of course.
Yes, unless the caller does not expect any more alerts after unregistering (I guess that's what you're saying?)
Quoted 3 lines
Bottom line is, while it may be critical that the IAM changes implied by register take place immediately, unregister is a different story.
It does feel a bit weird, though. But maybe I just have trouble grasping the design :/
Re: [PATCH v5 10/15] KVM: s390: add functions to (un)register GISC with GISA
Quoted 49 lines · On Tue, 8 Jan 2019 14:41:35 +0100 Cornelia Huck <[email protected]> wrote:
Quoted 46 lines · On Tue, 8 Jan 2019 14:36:13 +0100 Halil Pasic <[email protected]> wrote:
Quoted 29 lines · On Tue, 8 Jan 2019 11:34:44 +0100 Cornelia Huck <[email protected]> wrote:
Quoted 29 lines
Quoted 28 lines
Quoted 17 lines
Quoted 17 lines
(quoted diff) +16
-0+ spin_unlock(&kvm->arch.iam_ref_lock); + + return gib->nisc; +} +EXPORT_SYMBOL_GPL(kvm_s390_gisc_register); + +int kvm_s390_gisc_unregister(struct kvm *kvm, u32 gisc) +{ + int rc = 0; + + if (!kvm->arch.gib_in_use) + return -ENODEV; + if (gisc > MAX_ISC) + return -ERANGE; + + spin_lock(&kvm->arch.iam_ref_lock);Quoted 8 lines
Review
Quoted 8 lines
(quoted diff) +8
-0+ if (kvm->arch.iam_ref_count[gisc] == 0) { + rc = -EINVAL; + goto out; + } + kvm->arch.iam_ref_count[gisc]--; + if (kvm->arch.iam_ref_count[gisc] == 0) { + kvm->arch.iam &= ~(0x80 >> gisc); + set_iam(kvm->arch.gisa, kvm->arch.iam);Any chance of this function failing here? If yes, would there be any implications?
Quoted 5 lines
Quoted 1 line
It is the same here.
I'm not sure that I follow: This is the reverse operation (unregistering the gisc). Can we rely on
get_ipm()to do any fixup later? Is that a problem for the caller?IMHO gib alerts are all about not loosing initiative. I.e. avoiding substantially delayed delivery of interrupts due to vCPUs in wait state.
Thus we must avoid letting go before setting up stuff (gisa.iam under consideration of gisa ipm, gisa.next_alert, and gib.alert_list_origin) so that we can react on the next interrupt (if necessary).
Quoted 5 lines
Quoted 2 lines
On the other hand, getting more gisa alerts than necessary is not fatal -- better avoided if not too much hassle of course.
Yes, unless the caller does not expect any more alerts after unregistering (I guess that's what you're saying?)
Just to make sure, the caller, which I guess would probably be some sort of a driver, does not process the alerts. It is kvm that does.
But you are right, the properties of unregister need do be considered in the cleanup code (e.g. resets). Sure, we must avoid the process_gib_alert_list() (which is common, i.e. ain't tied to any VM) poking dead GISAs. Frankly, I've my hands full with the normal operation scenario, and I did not pay much attention to the cleanup yet.
Quoted 8 lines
Quoted 4 lines
Bottom line is, while it may be critical that the IAM changes implied by register take place immediately, unregister is a different story.
It does feel a bit weird, though. But maybe I just have trouble grasping the design :/
You are on the only one. I have extreme difficulties following some of the discussion here -- despite of the fact that I have quite some confidence in my understanding of the GISA/GIB architecture.
Regards,
Halil
PATCH v5 11/15 KVM: s390: restore IAM in get_ipm() when IPM is clean
In reply to Michael MuellerThe patch adds the parameter irq_flags and allows to restore the Interruption Alert Mask (IAM) in the GISA atomically while guaranteeing the IPM is clean.
The function returns the IPM of the GISA. If the returned value is 0x00 and the IRQ_FLAG_IAM was set, the IAM has been restored.
New irq flag:
IRQ_FLAG_IAM: When set, the IAM is restored if no ISC bit
is set in the IPM, i.e. no new airqs are pending. The test and restore operations are done atomically.
--- arch/s390/kvm/interrupt.c | 34 +++++++++++++++++++++++++++++----- 1 file changed, 29 insertions(+), 5 deletions(-)
arch/s390/kvm/interrupt.c +29 -5
@@ -37,6 +37,7 @@#define IRQ_FLAG_LOCAL 0x8000/* include local interruption pending mask */ #define IRQ_FLAG_FLOATING 0x4000/* include float interruption pending mask */ #define IRQ_FLAG_GISA 0x2000/* include GISA interruption pending mask */ +#define IRQ_FLAG_IAM 0x0080/* when set try to restore IAM */ #define IRQ_MASK_ALL (IRQ_FLAG_LOCAL | IRQ_FLAG_FLOATING | IRQ_FLAG_GISA)#define IRQ_MASK_NO_GISA (IRQ_MASK_ALL & ~IRQ_FLAG_GISA)
@@ -253,9 +254,32 @@ static inline void set_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gisc)set_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); } -static inline u8 get_ipm (struct kvm_s390_gisa *gisa) +static inline u8 get_ipm (struct kvm_s390_gisa *gisa,u16 irq_flags) { -return READ_ONCE (gisa->ipm); +u64 word, _word; +u8 ipm; + +if (!(irq_flags &IRQ_FLAG_IAM )) +return READ_ONCE (gisa->ipm); + +do { + word =READ_ONCE (gisa->u64.word[0 ]); + ipm = word >>24 ; +/* If the GISA is in the alert list, return the IPM. */ +if ((u64 )gisa != word >>32 ) +return ipm; +/* If the IPM is dirty, return the IPM. */ +if (ipm) +return ipm; +/* +* Try to restore the IAM or loop, if the IPM is dirty +* again or the GISA has been inserted into the alert list. +*/ + _word = (word & ~0xffUL ) | + container_of(gisa, struct sie_page2, gisa)->kvm->arch.iam; + }while (cmpxchg (&gisa->u64.word[0 ], word, _word) != _word); + +return 0 ; }static inline void clear_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc)
@@ -277,7 +301,7 @@ static inline unsigned long pending_irqs(struct kvm_vcpu *vcpu, u16 irq_flags)if (irq_flags &IRQ_FLAG_FLOATING ) pending_irqs |= vcpu->kvm->arch.float_int.pending_irqs;if (irq_flags &IRQ_FLAG_GISA ) - pending_irqs |=get_ipm (vcpu->kvm->arch.gisa) << + pending_irqs |=get_ipm (vcpu->kvm->arch.gisa, irq_flags) <<IRQ_PEND_IO_ISC_7 ;return pending_irqs; }
@@ -1574,7 +1598,7 @@ static int get_top_gisa_isc(struct kvm *kvm, u64 isc_mask, u32 schid)if (!kvm->arch.gisa) goto out; - active_mask = (isc_mask &get_ipm (kvm->arch.gisa) <<24 ) <<32 ; + active_mask = (isc_mask &get_ipm (kvm->arch.gisa,IRQ_FLAG_IAM ) <<24 ) <<32 ;while (active_mask) { isc =__fls (active_mask) ^ (BITS_PER_LONG -1 );if (tac_ipm_gisc (kvm->arch.gisa, isc))
@@ -2060,7 +2084,7 @@ static int get_all_floating_irqs(struct kvm *kvm, u8 __user *usrbuf, u64 len) max_irqs = len /sizeof (struct kvm_s390_irq );if (kvm->arch.gisa && -get_ipm (kvm->arch.gisa)) { +get_ipm (kvm->arch.gisa,IRQ_FLAG_IAM )) {for (i =0 ; i <=MAX_ISC ; i++) {if (n == max_irqs) {/* signal userspace to try again */
Signature · 1 line
2.13.4
Re: [PATCH v5 11/15] KVM: s390: restore IAM in get_ipm() when IPM is clean
Quoted 54 lines · Michael Mueller
The patch adds the parameter
irq_flagsand allows to restore the Interruption Alert Mask (IAM) in the GISA atomically while guaranteeing the IPM is clean.The function returns the IPM of the GISA. If the returned value is
0x00and theIRQ_FLAG_IAMwas set, the IAM has been restored.New irq flag:
IRQ_FLAG_IAM: When set, the IAM is restored if no ISC bit
is set in the IPM, i.e. no new airqs are pending. The test and restore operations are done atomically.Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 34 +++++++++++++++++++++++++++++----- 1 file changed, 29 insertions(+), 5 deletions(-)arch/s390/kvm/interrupt.c +17
-2@@ -37,6 +37,7 @@#define IRQ_FLAG_LOCAL 0x8000/* include local interruption pending mask */ #define IRQ_FLAG_FLOATING 0x4000/* include float interruption pending mask */ #define IRQ_FLAG_GISA 0x2000/* include GISA interruption pending mask */ +#define IRQ_FLAG_IAM 0x0080/* when set try to restore IAM */ #define IRQ_MASK_ALL (IRQ_FLAG_LOCAL | IRQ_FLAG_FLOATING | IRQ_FLAG_GISA)#define IRQ_MASK_NO_GISA (IRQ_MASK_ALL & ~IRQ_FLAG_GISA)@@ -253,9 +254,32 @@ static inline void set_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gisc)set_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); } -static inline u8 get_ipm (struct kvm_s390_gisa *gisa) +static inline u8 get_ipm (struct kvm_s390_gisa *gisa,u16 irq_flags) { -return READ_ONCE (gisa->ipm); +u64 word, _word; +u8 ipm; + +if (!(irq_flags &IRQ_FLAG_IAM )) +return READ_ONCE (gisa->ipm); + + do { + word = READ_ONCE(gisa->u64.word[0]); + ipm = word >> 24; + /* If the GISA is in the alert list, return the IPM. */ + if ((u64)gisa != word >> 32) + return ipm; + /* If the IPM is dirty, return the IPM. */ + if (ipm) + return ipm;
arch/s390/kvm/interrupt.c static inline void set_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gisc) · line 283
Quoted 10 lines
arch/s390/kvm/interrupt.c +9
-0+/* +* Try to restore the IAM or loop, if the IPM is dirty +* again or the GISA has been inserted into the alert list. +*/ + _word = (word & ~0xffUL ) | + container_of(gisa, struct sie_page2, gisa)->kvm->arch.iam; + }while (cmpxchg (&gisa->u64.word[0 ], word, _word) != _word); + +return 0 ; }
Personal opinion, but this function do more than just getting the IPM, shouldn't it be reflected in the function name?
Quoted 12 lines
arch/s390/kvm/interrupt.c +1
-1static inline void clear_ipm_gisc (struct kvm_s390_gisa *gisa,u32 gisc)@@ -277,7 +301,7 @@ static inline unsigned long pending_irqs(struct kvm_vcpu *vcpu, u16 irq_flags)if (irq_flags &IRQ_FLAG_FLOATING ) pending_irqs |= vcpu->kvm->arch.float_int.pending_irqs;if (irq_flags &IRQ_FLAG_GISA ) - pending_irqs |=get_ipm (vcpu->kvm->arch.gisa) << + pending_irqs |=get_ipm (vcpu->kvm->arch.gisa, irq_flags) <<IRQ_PEND_IO_ISC_7 ;return pending_irqs; }
arch/s390/kvm/interrupt.c static int get_top_gisa_isc(struct kvm *kvm, u64 isc_mask, u32 schid) · line 1601
Quoted 5 lines
arch/s390/kvm/interrupt.c +1
-1@@ -1574,7 +1598,7 @@ static int get_top_gisa_isc(struct kvm *kvm, u64 isc_mask, u32 schid)if (!kvm->arch.gisa) goto out; - active_mask = (isc_mask &get_ipm (kvm->arch.gisa) <<24 ) <<32 ; + active_mask = (isc_mask &get_ipm (kvm->arch.gisa,IRQ_FLAG_IAM ) <<24 ) <<32 ;
why do we need to set the IAM here?
arch/s390/kvm/interrupt.c static int get_all_floating_irqs(struct kvm *kvm, u8 __user *usrbuf, u64 len) · line 2087
Quoted 9 lines
arch/s390/kvm/interrupt.c +1
-1while (active_mask) { isc =__fls (active_mask) ^ (BITS_PER_LONG -1 );if (tac_ipm_gisc (kvm->arch.gisa, isc))@@ -2060,7 +2084,7 @@ static int get_all_floating_irqs(struct kvm *kvm, u8 __user *usrbuf, u64 len) max_irqs = len /sizeof (struct kvm_s390_irq );if (kvm->arch.gisa && -get_ipm (kvm->arch.gisa)) { +get_ipm (kvm->arch.gisa,IRQ_FLAG_IAM )) {
and here?
Quoted 4 lines
arch/s390/kvm/interrupt.c +0
-0for (i =0 ; i <=MAX_ISC ; i++) {if (n == max_irqs) {/* signal userspace to try again */
Signature · 2 lines
Pierre Morel Linux/KVM/QEMU in Böblingen - Germany
Re: [PATCH v5 11/15] KVM: s390: restore IAM in get_ipm() when IPM is clean
Quoted 58 lines · Pierre Morel
Quoted 57 lines · Michael Mueller
The patch adds the parameter
irq_flagsand allows to restore the Interruption Alert Mask (IAM) in the GISA atomically while guaranteeing the IPM is clean.The function returns the IPM of the GISA. If the returned value is
0x00and theIRQ_FLAG_IAMwas set, the IAM has been restored.New irq flag:
IRQ_FLAG_IAM: When set, the IAM is restored if no ISC bit is set in the IPM, i.e. no new airqs are pending. The test and restore operations are done atomically.Signed-off-by Michael Mueller <[email protected]>---
arch/s390/kvm/interrupt.c| 34 +++++++++++++++++++++++++++++-----1 file changed, 29 insertions(+), 5 deletions(-)arch/s390/kvm/interrupt.c +21
-2@@ -37,6 +37,7 @@ #define IRQ_FLAG_LOCAL 0x8000 /* include local interruption pending mask */ #define IRQ_FLAG_FLOATING 0x4000 /* include float interruption pending mask */ #define IRQ_FLAG_GISA 0x2000 /* include GISA interruption pending mask */ +#define IRQ_FLAG_IAM 0x0080/* when set try to restore IAM */ #define IRQ_MASK_ALL (IRQ_FLAG_LOCAL | IRQ_FLAG_FLOATING | IRQ_FLAG_GISA) #define IRQ_MASK_NO_GISA (IRQ_MASK_ALL & ~IRQ_FLAG_GISA)@@ -253,9 +254,32 @@ static inline void set_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gisc) set_bit_inv(IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); } -static inline u8 get_ipm (struct kvm_s390_gisa *gisa) +static inline u8 get_ipm (struct kvm_s390_gisa *gisa,u16 irq_flags) { - return READ_ONCE(gisa->ipm); + u64 word, _word; + u8 ipm; + + if (!(irq_flags & IRQ_FLAG_IAM)) + return READ_ONCE(gisa->ipm); + + do { + word = READ_ONCE(gisa->u64.word[0]); + ipm = word >> 24; + /* If the GISA is in the alert list, return the IPM. */ + if ((u64)gisa != word >> 32) + return ipm; + /* If the IPM is dirty, return the IPM. */ + if (ipm) + return ipm; + /* + * Try to restore the IAM or loop, if the IPM is dirty + * again or the GISA has been inserted into the alert list. + */
Quoted 8 lines
arch/s390/kvm/interrupt.c static inline void set_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gisc) · line 283
Quoted 6 lines
arch/s390/kvm/interrupt.c +5
-0+ _word = (word & ~0xffUL) | + container_of(gisa, struct sie_page2, gisa)->kvm->arch.iam; + } while (cmpxchg(&gisa->u64.word[0], word, _word) != _word); + + return 0; }Personal opinion, but this function do more than just getting the IPM, shouldn't it be reflected in the function name?
The main task of this function is to fetch the IPM. The additional functionality is expressed in the flags.
A call like get_ipm_try_iam_restore(kvm, 0) would be more confusing.
Quoted 13 lines
Quoted 13 lines
arch/s390/kvm/interrupt.c +1
-1static inline void clear_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gisc)@@ -277,7 +301,7 @@ static inline unsigned long pending_irqs(struct kvm_vcpu *vcpu, u16 irq_flags) if (irq_flags & IRQ_FLAG_FLOATING) pending_irqs |= vcpu->kvm->arch.float_int.pending_irqs; if (irq_flags & IRQ_FLAG_GISA) - pending_irqs |= get_ipm(vcpu->kvm->arch.gisa) << + pending_irqs |= get_ipm(vcpu->kvm->arch.gisa, irq_flags) << IRQ_PEND_IO_ISC_7; return pending_irqs;}
arch/s390/kvm/interrupt.c +0
-0@@ -1574,7 +1598,7 @@ static int get_top_gisa_isc(struct kvm *kvm, u64
Quoted 8 lines
arch/s390/kvm/interrupt.c static int get_top_gisa_isc(struct kvm *kvm, u64 isc_mask, u32 schid)
Quoted 6 lines
arch/s390/kvm/interrupt.c +1
-1isc_mask, u32 schid) if (!kvm->arch.gisa) goto out; - active_mask = (isc_mask & get_ipm(kvm->arch.gisa) << 24) << 32; + active_mask = (isc_mask & get_ipm(kvm->arch.gisa, IRQ_FLAG_IAM) <<24 ) <<32 ;why do we need to set the IAM here?
Yes, it should not be required, only when a vcpu goes into wait state.
Quoted 12 lines
Quoted 10 lines
arch/s390/kvm/interrupt.c +0
-0while (active_mask) { isc = __fls(active_mask) ^ (BITS_PER_LONG - 1);if (
tac_ipm_gisc(kvm->arch.gisa, isc))arch/s390/kvm/interrupt.c +0
-0@@ -2060,7 +2084,7 @@ static int get_all_floating_irqs(struct kvm *kvm, u8 __user *usrbuf, u64 len) max_irqs = len / sizeof(struct kvm_s390_irq);if (
kvm->arch.gisa&&
-get_ipm(kvm->arch.gisa)) {
+get_ipm(kvm->arch.gisa, IRQ_FLAG_IAM)) {and here?
Yes, it should not be required, only when a vcpu goes into wait state.
Quoted 7 lines
Quoted 5 lines
for (i = 0; i <=
MAX_ISC; i++) {
if (n ==max_irqs) {
/* signal userspace to try again */
Re: [PATCH v5 11/15] KVM: s390: restore IAM in get_ipm() when IPM is clean
In reply to Michael MuellerQuoted 51 lines · On Wed, 19 Dec 2018 20:17:52 +0100 Michael Mueller <[email protected]> wrote:
The patch adds the parameter
irq_flagsand allows to restore the Interruption Alert Mask (IAM) in the GISA atomically while guaranteeing the IPM is clean.The function returns the IPM of the GISA. If the returned value is
0x00and theIRQ_FLAG_IAMwas set, the IAM has been restored.New irq flag:
IRQ_FLAG_IAM: When set, the IAM is restored if no ISC bit
is set in the IPM, i.e. no new airqs are pending. The test and restore operations are done atomically.Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 34 +++++++++++++++++++++++++++++----- 1 file changed, 29 insertions(+), 5 deletions(-)arch/s390/kvm/interrupt.c +14
-2@@ -37,6 +37,7 @@#define IRQ_FLAG_LOCAL 0x8000/* include local interruption pending mask */ #define IRQ_FLAG_FLOATING 0x4000/* include float interruption pending mask */ #define IRQ_FLAG_GISA 0x2000/* include GISA interruption pending mask */ +#define IRQ_FLAG_IAM 0x0080/* when set try to restore IAM */ #define IRQ_MASK_ALL (IRQ_FLAG_LOCAL | IRQ_FLAG_FLOATING | IRQ_FLAG_GISA)#define IRQ_MASK_NO_GISA (IRQ_MASK_ALL & ~IRQ_FLAG_GISA)@@ -253,9 +254,32 @@ static inline void set_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gisc)set_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); } -static inline u8 get_ipm (struct kvm_s390_gisa *gisa) +static inline u8 get_ipm (struct kvm_s390_gisa *gisa,u16 irq_flags) { -return READ_ONCE (gisa->ipm); +u64 word, _word; +u8 ipm; + +if (!(irq_flags &IRQ_FLAG_IAM )) +return READ_ONCE (gisa->ipm); + + do { + word = READ_ONCE(gisa->u64.word[0]); + ipm = word >> 24; + /* If the GISA is in the alert list, return the IPM. */ + if ((u64)gisa != word >> 32) + return ipm;
arch/s390/kvm/interrupt.c static inline void set_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gisc) · line 280
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+/* If the IPM is dirty, return the IPM. */ +if (ipm) +return ipm; +/* +* Try to restore the IAM or loop, if the IPM is dirty +* again or the GISA has been inserted into the alert list. +*/ + _word = (word & ~0xffUL ) | + container_of(gisa, struct sie_page2, gisa)->kvm->arch.iam; + }while (cmpxchg (&gisa->u64.word[0 ], word, _word) != _word);
^^
Shouldn't this rather be:
} while (cmpxchg(&gisa->u64.word[0], word, _word) != word);
Regards,
Halil
Quoted 3 lines
arch/s390/kvm/interrupt.c +2
-0+ +return 0 ; }
Re: [PATCH v5 11/15] KVM: s390: restore IAM in get_ipm() when IPM is clean
Quoted 66 lines · Halil Pasic
Quoted 51 lines · On Wed, 19 Dec 2018 20:17:52 +0100 Michael Mueller <[email protected]> wrote:
The patch adds the parameter
irq_flagsand allows to restore the Interruption Alert Mask (IAM) in the GISA atomically while guaranteeing the IPM is clean.The function returns the IPM of the GISA. If the returned value is
0x00and theIRQ_FLAG_IAMwas set, the IAM has been restored.New irq flag:
IRQ_FLAG_IAM: When set, the IAM is restored if no ISC bit
is set in the IPM, i.e. no new airqs are pending. The test and restore operations are done atomically.Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 34 +++++++++++++++++++++++++++++----- 1 file changed, 29 insertions(+), 5 deletions(-)arch/s390/kvm/interrupt.c +14
-2@@ -37,6 +37,7 @@#define IRQ_FLAG_LOCAL 0x8000/* include local interruption pending mask */ #define IRQ_FLAG_FLOATING 0x4000/* include float interruption pending mask */ #define IRQ_FLAG_GISA 0x2000/* include GISA interruption pending mask */ +#define IRQ_FLAG_IAM 0x0080/* when set try to restore IAM */ #define IRQ_MASK_ALL (IRQ_FLAG_LOCAL | IRQ_FLAG_FLOATING | IRQ_FLAG_GISA)#define IRQ_MASK_NO_GISA (IRQ_MASK_ALL & ~IRQ_FLAG_GISA)@@ -253,9 +254,32 @@ static inline void set_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gisc)set_bit_inv (IPM_BIT_OFFSET + gisc, (unsigned long *) gisa); } -static inline u8 get_ipm (struct kvm_s390_gisa *gisa) +static inline u8 get_ipm (struct kvm_s390_gisa *gisa,u16 irq_flags) { -return READ_ONCE (gisa->ipm); +u64 word, _word; +u8 ipm; + +if (!(irq_flags &IRQ_FLAG_IAM )) +return READ_ONCE (gisa->ipm); + + do { + word = READ_ONCE(gisa->u64.word[0]); + ipm = word >> 24; + /* If the GISA is in the alert list, return the IPM. */ + if ((u64)gisa != word >> 32) + return ipm;arch/s390/kvm/interrupt.c static inline void set_ipm_gisc(struct kvm_s390_gisa *gisa, u32 gisc) · line 280
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+/* If the IPM is dirty, return the IPM. */ +if (ipm) +return ipm; +/* +* Try to restore the IAM or loop, if the IPM is dirty +* again or the GISA has been inserted into the alert list. +*/ + _word = (word & ~0xffUL ) | + container_of(gisa, struct sie_page2, gisa)->kvm->arch.iam; + }while (cmpxchg (&gisa->u64.word[0 ], word, _word) != _word);^^
Quoted 2 lines
Shouldn't this rather be:
} while (cmpxchg(&gisa->u64.word[0], word, _word) != word);
Yes, the comparison needs to be done against the old value of the memory region.
Quoted 6 lines
Regards,
HalilQuoted 3 lines
arch/s390/kvm/interrupt.c +2
-0+ +return 0 ; }
PATCH v5 12/15 KVM: s390: do not restore IAM immediately before SIE entry
In reply to Michael MuellerThe IAM shall no be restored when deliverable interruptions are delivered to vcpus by means of the PSW swap mechanism. That would trigger the GIB alert millicode although we know that SIE will be able to handle the pending interruption on entry.
--- arch/s390/kvm/interrupt.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
arch/s390/kvm/interrupt.c +4 -4
@@ -328,11 +328,11 @@ static unsigned long disable_iscs(struct kvm_vcpu *vcpu,return active_mask; } -static unsigned long deliverable_irqs (struct kvm_vcpu *vcpu) +static unsigned long deliverable_irqs (struct kvm_vcpu *vcpu,u16 irq_flags) {unsigned long active_mask; - active_mask =pending_irqs (vcpu,IRQ_MASK_ALL ); + active_mask =pending_irqs (vcpu, irq_flags);if (!active_mask)return 0 ;
@@ -1090,7 +1090,7 @@ int kvm_s390_ext_call_pending(struct kvm_vcpu *vcpu)int kvm_s390_vcpu_has_irq (struct kvm_vcpu *vcpu,int exclude_stop) { -if (deliverable_irqs (vcpu)) +if (deliverable_irqs (vcpu,IRQ_MASK_ALL |IRQ_FLAG_IAM ))return 1 ;if (kvm_cpu_has_pending_timer (vcpu))
@@ -1262,7 +1262,7 @@ int __must_check kvm_s390_deliver_pending_interrupts(struct kvm_vcpu *vcpu)if (cpu_timer_irq_pending (vcpu))set_bit (IRQ_PEND_EXT_CPU_TIMER , &li->pending_irqs); -while ((irqs =deliverable_irqs (vcpu)) && !rc) { +while ((irqs =deliverable_irqs (vcpu,IRQ_MASK_ALL )) && !rc) {/* bits are in the reverse order of interrupt priority */ irq_type =find_last_bit (&irqs,IRQ_PEND_COUNT );switch (irq_type) {
Signature · 1 line
2.13.4
Re: [PATCH v5 12/15] KVM: s390: do not restore IAM immediately before SIE entry
Quoted 29 lines · Michael Mueller
The IAM shall no be restored when deliverable interruptions are delivered to vcpus by means of the PSW swap mechanism. That would trigger the GIB alert millicode although we know that SIE will be able to handle the pending interruption on entry.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)arch/s390/kvm/interrupt.c +2
-2@@ -328,11 +328,11 @@ static unsigned long disable_iscs(struct kvm_vcpu *vcpu,return active_mask; } -static unsigned long deliverable_irqs (struct kvm_vcpu *vcpu) +static unsigned long deliverable_irqs (struct kvm_vcpu *vcpu,u16 irq_flags) {unsigned long active_mask; - active_mask =pending_irqs (vcpu,IRQ_MASK_ALL ); + active_mask =pending_irqs (vcpu, irq_flags);if (!active_mask)return 0 ;
arch/s390/kvm/interrupt.c int kvm_s390_ext_call_pending(struct kvm_vcpu *vcpu) · line 1093
Quoted 5 lines
arch/s390/kvm/interrupt.c +1
-1@@ -1090,7 +1090,7 @@ int kvm_s390_ext_call_pending(struct kvm_vcpu *vcpu)int kvm_s390_vcpu_has_irq (struct kvm_vcpu *vcpu,int exclude_stop) { -if (deliverable_irqs (vcpu)) +if (deliverable_irqs (vcpu,IRQ_MASK_ALL |IRQ_FLAG_IAM ))
Why do we need to restore IAM here?
Signature · 2 lines
Pierre Morel Linux/KVM/QEMU in Böblingen - Germany
Re: [PATCH v5 12/15] KVM: s390: do not restore IAM immediately before SIE entry
Quoted 28 lines · Pierre Morel
Quoted 27 lines · Michael Mueller
The IAM shall no be restored when deliverable interruptions are delivered to vcpus by means of the PSW swap mechanism. That would trigger the GIB alert millicode although we know that SIE will be able to handle the pending interruption on entry.
Signed-off-by Michael Mueller <[email protected]>---
arch/s390/kvm/interrupt.c| 8 ++++----1 file changed, 4 insertions(+), 4 deletions(-)arch/s390/kvm/interrupt.c +2
-2@@ -328,11 +328,11 @@ static unsigned long disable_iscs(structkvm_vcpu *vcpu, return active_mask; } -static unsigned long deliverable_irqs (struct kvm_vcpu *vcpu) +static unsigned long deliverable_irqs (struct kvm_vcpu *vcpu,u16 irq_flags) { unsigned long active_mask; - active_mask = pending_irqs(vcpu, IRQ_MASK_ALL); + active_mask = pending_irqs(vcpu, irq_flags); if (!active_mask) return 0;
Quoted 8 lines
arch/s390/kvm/interrupt.c int kvm_s390_ext_call_pending(struct kvm_vcpu · line 1093
Quoted 6 lines
arch/s390/kvm/interrupt.c +1
-1@@ -1090,7 +1090,7 @@ int kvm_s390_ext_call_pending(struct kvm_vcpu *vcpu) int kvm_s390_vcpu_has_irq(struct kvm_vcpu *vcpu, int exclude_stop) { - if (deliverable_irqs(vcpu)) + if (deliverable_irqs(vcpu, IRQ_MASK_ALL | IRQ_FLAG_IAM))Why do we need to restore IAM here?
please see kvm_s390_handle_wait()
It calls kvm_arch_vcpu_runnable() / kvm_s390_vcpu_has_irq()
That's the place where we want the IAM to be restored when no ISC is pending in the IPM anymore.
Quoted 3 lines
PATCH v5 13/15 KVM: s390: add function process_gib_alert_list()
In reply to Michael MuellerThis function processes the Gib Alert List (GAL). It is required to run when either a gib alert interruption has been received or a gisa that is in the alert list is cleared or dropped.
The GAL is build up by millicode, when the respective ISC bit is set in the Interruption Alert Mask (IAM) and an interruption of that class is observed.
--- arch/s390/kvm/interrupt.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 140 insertions(+)
arch/s390/kvm/interrupt.c +140 -0
@@ -2941,6 +2941,146 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len)return n; } +static int __try_airqs_kick (struct kvm *kvm,u8 ipm) +{ +struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; +struct kvm_vcpu *vcpu =NULL , *kick_vcpu[MAX_ISC +1 ]; +int online_vcpus =atomic_read (&kvm->online_vcpus); +u8 ioint_mask, isc_mask, kick_mask =0x00 ; +int vcpu_id, kicked =0 ; + +/* Loop over vcpus in WAIT state. */ +for (vcpu_id =find_first_bit (fi->idle_mask, online_vcpus); +/* Until all pending ISCs have a vcpu open for airqs. */ + (~kick_mask & ipm) && vcpu_id < online_vcpus; + vcpu_id =find_next_bit (fi->idle_mask, online_vcpus, vcpu_id)) { + vcpu =kvm_get_vcpu (kvm, vcpu_id); +if (psw_ioint_disabled (vcpu)) +continue ; + ioint_mask = (u8)(vcpu->arch.sie_block->gcr[6 ] >>24 ); +for (isc_mask =0x80 ; isc_mask; isc_mask >>=1 ) { +/* ISC pending in IPM ? */ +if (!(ipm & isc_mask)) +continue ; +/* vcpu for this ISC already found ? */ +if (kick_mask & isc_mask) +continue ; +/* vcpu open for airq of this ISC ? */ +if (!(ioint_mask & isc_mask)) +continue ; +/* use this vcpu (for all ISCs in ioint_mask) */ + kick_mask |= ioint_mask; + kick_vcpu[kicked++] = vcpu; + } + } + +if (vcpu && ~kick_mask & ipm) +VM_EVENT (kvm,4 ,"gib alert undeliverable isc mask 0x%02x" , + ~kick_mask & ipm); + +for (vcpu_id =0 ; vcpu_id < kicked; vcpu_id++) +kvm_s390_vcpu_wakeup (kick_vcpu[vcpu_id]); + +return (online_vcpus !=0 ) ? kicked : -ENODEV ; +} + +static void __floating_airqs_kick (struct kvm *kvm) +{ +struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; +int online_vcpus, kicked; +u8 ipm_t0, ipm; + +/* Get IPM and return if clean, IAM has been restored. */ + ipm =get_ipm (kvm->arch.gisa,IRQ_FLAG_IAM ); +if (!ipm) +return ; +retry: + ipm_t0 = ipm; + +/* Try to kick some vcpus in WAIT state. */ + kicked =__try_airqs_kick (kvm, ipm); +if (kicked <0 ) +return ; + +/* Get IPM and return if clean, IAM has been restored. */ + ipm =get_ipm (kvm->arch.gisa,IRQ_FLAG_IAM ); +if (!ipm) +return ; + +/* Start over, if new ISC bits are pending in IPM. */ +if ((ipm_t0 ^ ipm) & ~ipm_t0) + goto retry; + +/* +* Return as we just kicked at least one vcpu in WAIT state +* open for airqs. The IAM will be restored latest when one +* of them goes into WAIT or STOP state. +*/ +if (kicked >0 ) +return ; + +/* +* No vcpu was kicked either because no vcpu was in WAIT state +* or none of the vcpus in WAIT state are open for airqs. +* Return immediately if no vcpus are in WAIT state. +* There are vcpus in RUN state. They will process the airqs +* if not closed for airqs as well. In that case the system will +* delay airqs until a vcpu decides to take airqs again. +*/ + online_vcpus =atomic_read (&kvm->online_vcpus); +if (!bitmap_weight (fi->idle_mask, online_vcpus)) +return ; + +/* +* None of the vcpus in WAIT state take airqs and we might +* have no running vcpus as at least one vcpu is in WAIT state +* and IPM is dirty. +*/ +set_iam (kvm->arch.gisa, kvm->arch.iam); +} + +#define NULL_GISA_ADDR 0x00000000UL +#define NONE_GISA_ADDR 0x00000001UL +#define GISA_ADDR_MASK 0xfffff000UL + +static void __maybe_unused process_gib_alert_list (void ) +{ +u32 final, next_alert, origin =0UL ; +struct kvm_s390_gisa *gisa; +struct kvm *kvm; + +do { +/* +* If the NONE_GISA_ADDR is still stored in the alert list +* origin, we will leave the outer loop. No further GISA has +* been added to the alert list by millicode while processing +* the current alert list. +*/ + final = (origin &NONE_GISA_ADDR ); +/* +* Cut off the alert list and store the NONE_GISA_ADDR in the +* alert list origin to avoid further GAL interruptions. +* A new alert list can be build up by millicode in parallel +* for guests not in the yet cut-off alert list. When in the +* final loop, store the NULL_GISA_ADDR instead. This will re- +* enable GAL interruptions on the host again. +*/ + origin =xchg (&gib->alert_list_origin, + (!final) ?NONE_GISA_ADDR :NULL_GISA_ADDR ); +/* Loop through the just cut-off alert list. */ +while (origin &GISA_ADDR_MASK ) { + gisa = (struct kvm_s390_gisa *)(u64 )origin; + next_alert = gisa->next_alert; +/* Unlink the GISA from the alert list. */ + gisa->next_alert = origin; + kvm = container_of(gisa, struct sie_page2, gisa)->kvm; +/* Kick suitable vcpus */ +__floating_airqs_kick (kvm); + origin = next_alert; + } + }while (!final); +} +static void nullify_gisa (struct kvm_s390_gisa *gisa) {memset (gisa,0 ,sizeof (struct kvm_s390_gisa ));
Signature · 1 line
2.13.4
Re: [PATCH v5 13/15] KVM: s390: add function process_gib_alert_list()
Quoted 18 lines · Michael Mueller
This function processes the Gib Alert List (GAL). It is required to run when either a gib alert interruption has been received or a gisa that is in the alert list is cleared or dropped.
The GAL is build up by millicode, when the respective ISC bit is set in the Interruption Alert Mask (IAM) and an interruption of that class is observed.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 140 insertions(+)arch/s390/kvm/interrupt.c +0
-0
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2944
Quoted 4 lines
arch/s390/kvm/interrupt.c +1
-0@@ -2941,6 +2941,146 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len)return n; } +static int __try_airqs_kick (struct kvm *kvm,u8 ipm)
static inline ?
Quoted 32 lines
arch/s390/kvm/interrupt.c +32
-0+{ +struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; +struct kvm_vcpu *vcpu =NULL , *kick_vcpu[MAX_ISC +1 ]; +int online_vcpus =atomic_read (&kvm->online_vcpus); +u8 ioint_mask, isc_mask, kick_mask =0x00 ; +int vcpu_id, kicked =0 ; + +/* Loop over vcpus in WAIT state. */ +for (vcpu_id =find_first_bit (fi->idle_mask, online_vcpus); +/* Until all pending ISCs have a vcpu open for airqs. */ + (~kick_mask & ipm) && vcpu_id < online_vcpus; + vcpu_id =find_next_bit (fi->idle_mask, online_vcpus, vcpu_id)) { + vcpu =kvm_get_vcpu (kvm, vcpu_id); +if (psw_ioint_disabled (vcpu)) +continue ; + ioint_mask = (u8)(vcpu->arch.sie_block->gcr[6 ] >>24 ); +for (isc_mask =0x80 ; isc_mask; isc_mask >>=1 ) { +/* ISC pending in IPM ? */ +if (!(ipm & isc_mask)) +continue ; +/* vcpu for this ISC already found ? */ +if (kick_mask & isc_mask) +continue ; +/* vcpu open for airq of this ISC ? */ +if (!(ioint_mask & isc_mask)) +continue ; +/* use this vcpu (for all ISCs in ioint_mask) */ + kick_mask |= ioint_mask; > + kick_vcpu[kicked++] = vcpu; + } + } + +if (vcpu && ~kick_mask & ipm)
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2987
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+VM_EVENT (kvm,4 ,"gib alert undeliverable isc mask 0x%02x" , + ~kick_mask & ipm); + +for (vcpu_id =0 ; vcpu_id < kicked; vcpu_id++) +kvm_s390_vcpu_wakeup (kick_vcpu[vcpu_id]); + +return (online_vcpus !=0 ) ? kicked : -ENODEV ; +} + +static void __floating_airqs_kick (struct kvm *kvm)
static inline ?
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2994
Quoted 7 lines
arch/s390/kvm/interrupt.c +7
-0+{ +struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; +int online_vcpus, kicked; +u8 ipm_t0, ipm; + +/* Get IPM and return if clean, IAM has been restored. */ + ipm =get_ipm (kvm->arch.gisa,IRQ_FLAG_IAM );
If we do not get an IPM here, it must have been stolen by the firmware for delivery to the guest. Then why restoring the IAM?
Or do I miss something?
Quoted 35 lines
arch/s390/kvm/interrupt.c +35
-0+if (!ipm) +return ; +retry: + ipm_t0 = ipm; + +/* Try to kick some vcpus in WAIT state. */ + kicked =__try_airqs_kick (kvm, ipm); +if (kicked <0 ) +return ; + +/* Get IPM and return if clean, IAM has been restored. */ + ipm =get_ipm (kvm->arch.gisa,IRQ_FLAG_IAM ); +if (!ipm) +return ; + +/* Start over, if new ISC bits are pending in IPM. */ +if ((ipm_t0 ^ ipm) & ~ipm_t0) + goto retry; + +/* +* Return as we just kicked at least one vcpu in WAIT state +* open for airqs. The IAM will be restored latest when one +* of them goes into WAIT or STOP state. +*/ +if (kicked >0 ) +return ; + +/* +* No vcpu was kicked either because no vcpu was in WAIT state +* or none of the vcpus in WAIT state are open for airqs. +* Return immediately if no vcpus are in WAIT state. +* There are vcpus in RUN state. They will process the airqs +* if not closed for airqs as well. In that case the system will +* delay airqs until a vcpu decides to take airqs again. +*/
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3039
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+ online_vcpus =atomic_read (&kvm->online_vcpus); +if (!bitmap_weight (fi->idle_mask, online_vcpus)) +return ; + +/* +* None of the vcpus in WAIT state take airqs and we might +* have no running vcpus as at least one vcpu is in WAIT state +* and IPM is dirty. +*/ +set_iam (kvm->arch.gisa, kvm->arch.iam);
I do not understand why we need to set IAM here.
The interrupt will be delivered by the firmware as soon as the PSW or CR6 is changed by any vCPU. ...and if this does not happen we can not deliver the interrupt anyway.
Quoted 26 lines
arch/s390/kvm/interrupt.c +26
-0+} + +#define NULL_GISA_ADDR 0x00000000UL +#define NONE_GISA_ADDR 0x00000001UL +#define GISA_ADDR_MASK 0xfffff000UL + +static void __maybe_unused process_gib_alert_list (void ) +{ +u32 final, next_alert, origin =0UL ; +struct kvm_s390_gisa *gisa; +struct kvm *kvm; + + do { + /* + * If the NONE_GISA_ADDR is still stored in the alert list + * origin, we will leave the outer loop. No further GISA has + * been added to the alert list by millicode while processing + * the current alert list. + */ + final = (origin & NONE_GISA_ADDR); + /* + * Cut off the alert list and store the NONE_GISA_ADDR in the + * alert list origin to avoid further GAL interruptions. + * A new alert list can be build up by millicode in parallel + * for guests not in the yet cut-off alert list. When in the + * final loop, store the NULL_GISA_ADDR instead. This will re-
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3075
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+* enable GAL interruptions on the host again. +*/ + origin =xchg (&gib->alert_list_origin, + (!final) ?NONE_GISA_ADDR :NULL_GISA_ADDR ); +/* Loop through the just cut-off alert list. */ +while (origin &GISA_ADDR_MASK ) { + gisa = (struct kvm_s390_gisa *)(u64 )origin; + next_alert = gisa->next_alert; +/* Unlink the GISA from the alert list. */ + gisa->next_alert = origin;
AFAIU this enable GISA interrupt for the guest...
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3078
Quoted 3 lines
arch/s390/kvm/interrupt.c +3
-0+ kvm = container_of(gisa, struct sie_page2, gisa)->kvm; +/* Kick suitable vcpus */ +__floating_airqs_kick (kvm);
...and here we kick a VCPU for the guest.
Logically I would do it in the otherway, first kicking the vCPU then enabling the GISA interruption again.
If the IPM bit is cleared by the firmware during delivering the interrupt to the guest before we enter get_ipm() called by __floating_airqs_kick() we will set the IAM despite we have a running CPU handling the IRQ. In the worst case we can also set the IAM with the GISA in the alert list.
Or we must accept that the firmware can deliver the IPM as soon as we reset the GISA next field.
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3086
Quoted 9 lines
arch/s390/kvm/interrupt.c +5
-0+ origin = next_alert; + } + }while (!final); +} +static void nullify_gisa (struct kvm_s390_gisa *gisa) {memset (gisa,0 ,sizeof (struct kvm_s390_gisa ));
I think that avoiding to restore the IAM during the call to get_ipm inside __floating_airqs_kick() would good.
If you agree, with that:
Signature · 2 lines
Pierre Morel Linux/KVM/QEMU in Böblingen - Germany
Re: [PATCH v5 13/15] KVM: s390: add function process_gib_alert_list()
Quoted 73 lines · Pierre Morel
Quoted 19 lines · Michael Mueller
This function processes the Gib Alert List (GAL). It is required to run when either a gib alert interruption has been received or a gisa that is in the alert list is cleared or dropped.
The GAL is build up by millicode, when the respective ISC bit is set in the Interruption Alert Mask (IAM) and an interruption of that class is observed.
Signed-off-by Michael Mueller <[email protected]>---
arch/s390/kvm/interrupt.c| 140
++++++++++++++++++++++++++++++++++++++++++++++1 file changed, 140 insertions(+)arch/s390/kvm/interrupt.c +0
-0arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu · line 2944
Quoted 4 lines
arch/s390/kvm/interrupt.c +1
-0@@ -2941,6 +2941,146 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) return n; } +static int __try_airqs_kick (struct kvm *kvm,u8 ipm)static inline ?
Quoted 34 lines
arch/s390/kvm/interrupt.c +32
-0+{ + struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; + struct kvm_vcpu *vcpu = NULL, *kick_vcpu[MAX_ISC + 1]; + int online_vcpus = atomic_read(&kvm->online_vcpus); + u8 ioint_mask, isc_mask, kick_mask = 0x00; + int vcpu_id, kicked = 0; + + /* Loop over vcpus in WAIT state. */ + for (vcpu_id = find_first_bit(fi->idle_mask, online_vcpus); + /* Until all pending ISCs have a vcpu open for airqs. */ + (~kick_mask & ipm) && vcpu_id < online_vcpus; + vcpu_id = find_next_bit(fi->idle_mask, online_vcpus, vcpu_id)) { + vcpu = kvm_get_vcpu(kvm, vcpu_id); + if (psw_ioint_disabled(vcpu)) + continue; + ioint_mask = (u8)(vcpu->arch.sie_block->gcr[6] >> 24); + for (isc_mask = 0x80; isc_mask; isc_mask >>= 1) { + /* ISC pending in IPM ? */ + if (!(ipm & isc_mask)) + continue; + /* vcpu for this ISC already found ? */ + if (kick_mask & isc_mask) + continue; + /* vcpu open for airq of this ISC ? */ + if (!(ioint_mask & isc_mask)) + continue; + /* use this vcpu (for all ISCs in ioint_mask) */ + kick_mask |= ioint_mask; > + kick_vcpu[kicked++] = vcpu; + } + } + + if (vcpu && ~kick_mask & ipm)arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2987
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+ VM_EVENT(kvm, 4, "gib alert undeliverable isc mask 0x%02x", + ~kick_mask & ipm); + + for (vcpu_id = 0; vcpu_id < kicked; vcpu_id++) + kvm_s390_vcpu_wakeup(kick_vcpu[vcpu_id]); + + return (online_vcpus != 0) ? kicked : -ENODEV; +} + +static void __floating_airqs_kick (struct kvm *kvm)static inline ?
Quoted 9 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2994
Quoted 7 lines
arch/s390/kvm/interrupt.c +7
-0+{ + struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; + int online_vcpus, kicked; + u8 ipm_t0, ipm; + + /* Get IPM and return if clean, IAM has been restored. */ + ipm = get_ipm(kvm->arch.gisa, IRQ_FLAG_IAM);If we do not get an IPM here, it must have been stolen by the firmware for delivery to the guest.
Yes, a running SIE instance took it before we were able to. But is it still running now? It could have gone to WAIT before we see that the IPM is clean. Then it was restored already. Otherwise, it is still running and will go WAIT and then restore the IAM.
I will do some tests on this.
Quoted 84 lines
Then why restoring the IAM?
Or do I miss something?
Quoted 35 lines
arch/s390/kvm/interrupt.c +35
-0+ if (!ipm) + return; +retry: + ipm_t0 = ipm; + + /* Try to kick some vcpus in WAIT state. */ + kicked = __try_airqs_kick(kvm, ipm); + if (kicked < 0) + return; + + /* Get IPM and return if clean, IAM has been restored. */ + ipm = get_ipm(kvm->arch.gisa, IRQ_FLAG_IAM); + if (!ipm) + return; + + /* Start over, if new ISC bits are pending in IPM. */ + if ((ipm_t0 ^ ipm) & ~ipm_t0) + goto retry; + + /* + * Return as we just kicked at least one vcpu in WAIT state + * open for airqs. The IAM will be restored latest when one + * of them goes into WAIT or STOP state. + */ + if (kicked > 0) + return; + + /* +* No vcpu was kicked either because no vcpu was in WAIT state +* or none of the vcpus in WAIT state are open for airqs. +* Return immediately if no vcpus are in WAIT state. +* There are vcpus in RUN state. They will process the airqs +* if not closed for airqs as well. In that case the system will +* delay airqs until a vcpu decides to take airqs again. +*/ arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3039
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+ online_vcpus = atomic_read(&kvm->online_vcpus); + if (!bitmap_weight(fi->idle_mask, online_vcpus)) + return; + + /* + * None of the vcpus in WAIT state take airqs and we might + * have no running vcpus as at least one vcpu is in WAIT state + * and IPM is dirty. + */ + set_iam(kvm->arch.gisa, kvm->arch.iam);I do not understand why we need to set IAM here.
The interrupt will be delivered by the firmware as soon as the PSW or CR6 is changed by any vCPU. ...and if this does not happen we can not deliver the interrupt anyway.Quoted 30 lines
arch/s390/kvm/interrupt.c +30
-0+} + +#define NULL_GISA_ADDR 0x00000000UL +#define NONE_GISA_ADDR 0x00000001UL +#define GISA_ADDR_MASK 0xfffff000UL + +static void __maybe_unused process_gib_alert_list (void ) +{ + u32 final, next_alert, origin = 0UL; + struct kvm_s390_gisa *gisa; + struct kvm *kvm; + + do { + /* + * If the NONE_GISA_ADDR is still stored in the alert list + * origin, we will leave the outer loop. No further GISA has + * been added to the alert list by millicode while processing + * the current alert list. + */ + final = (origin & NONE_GISA_ADDR); + /* + * Cut off the alert list and store the NONE_GISA_ADDR in the + * alert list origin to avoid further GAL interruptions. + * A new alert list can be build up by millicode in parallel + * for guests not in the yet cut-off alert list. When in the + * final loop, store the NULL_GISA_ADDR instead. This will re- + * enable GAL interruptions on the host again. + */ + origin = xchg(&gib->alert_list_origin, + (!final) ? NONE_GISA_ADDR : NULL_GISA_ADDR);
Quoted 8 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3075
Quoted 6 lines
arch/s390/kvm/interrupt.c +6
-0+ /* Loop through the just cut-off alert list. */ + while (origin & GISA_ADDR_MASK) { + gisa = (struct kvm_s390_gisa *)(u64)origin; + next_alert = gisa->next_alert; + /* Unlink the GISA from the alert list. */ + gisa->next_alert = origin;AFAIU this enable GISA interrupt for the guest...
Only together with the IAM being set what could happen if __floating_airqs_kick() calls get_ipm and the IPM is clean already. :(
Quoted 10 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3078
Quoted 4 lines
arch/s390/kvm/interrupt.c +3
-0+ kvm = container_of(gisa, struct sie_page2, gisa)->kvm; + /* Kick suitable vcpus */ + __floating_airqs_kick(kvm);...and here we kick a VCPU for the guest.
Logically I would do it in the otherway, first kicking the vCPU then enabling the GISA interruption again.
If the IPM bit is cleared by the firmware during delivering the interrupt to the guest before we enter
get_ipm()called by__floating_airqs_kick()we will set the IAM despite we have a running CPU handling the IRQ.
I will move the unlink below the kick that will assure get_ipm will never take the IAM restore path.
Quoted 2 lines
In the worst case we can also set the IAM with the GISA in the alert list.
Or we must accept that the firmware can deliver the IPM as soon as we reset the GISA next field.
See statement above.
Quoted 18 lines
Quoted 10 lines
arch/s390/kvm/interrupt.c +5
-0+ origin = next_alert; + } + } while (!final); +} + static void nullify_gisa(struct kvm_s390_gisa *gisa){
memset(gisa, 0, sizeof(struct kvm_s390_gisa));I think that avoiding to restore the IAM during the call to
get_ipminside__floating_airqs_kick()would good.If you agree, with that:
Reviewed-by Pierre Morel<[email protected]>
Re: [PATCH v5 13/15] KVM: s390: add function process_gib_alert_list()
Quoted 206 lines · On Mon, 7 Jan 2019 20:18:03 +0100 Michael Mueller <[email protected]> wrote:
Quoted 73 lines · Pierre Morel
Quoted 19 lines · Michael Mueller
This function processes the Gib Alert List (GAL). It is required to run when either a gib alert interruption has been received or a gisa that is in the alert list is cleared or dropped.
The GAL is build up by millicode, when the respective ISC bit is set in the Interruption Alert Mask (IAM) and an interruption of that class is observed.
Signed-off-by Michael Mueller <[email protected]>---
arch/s390/kvm/interrupt.c| 140
++++++++++++++++++++++++++++++++++++++++++++++1 file changed, 140 insertions(+)arch/s390/kvm/interrupt.c +0
-0arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu · line 2944
Quoted 4 lines
arch/s390/kvm/interrupt.c +1
-0@@ -2941,6 +2941,146 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) return n; } +static int __try_airqs_kick (struct kvm *kvm,u8 ipm)static inline ?
Quoted 34 lines
arch/s390/kvm/interrupt.c +32
-0+{ + struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; + struct kvm_vcpu *vcpu = NULL, *kick_vcpu[MAX_ISC + 1]; + int online_vcpus = atomic_read(&kvm->online_vcpus); + u8 ioint_mask, isc_mask, kick_mask = 0x00; + int vcpu_id, kicked = 0; + + /* Loop over vcpus in WAIT state. */ + for (vcpu_id = find_first_bit(fi->idle_mask, online_vcpus); + /* Until all pending ISCs have a vcpu open for airqs. */ + (~kick_mask & ipm) && vcpu_id < online_vcpus; + vcpu_id = find_next_bit(fi->idle_mask, online_vcpus, vcpu_id)) { + vcpu = kvm_get_vcpu(kvm, vcpu_id); + if (psw_ioint_disabled(vcpu)) + continue; + ioint_mask = (u8)(vcpu->arch.sie_block->gcr[6] >> 24); + for (isc_mask = 0x80; isc_mask; isc_mask >>= 1) { + /* ISC pending in IPM ? */ + if (!(ipm & isc_mask)) + continue; + /* vcpu for this ISC already found ? */ + if (kick_mask & isc_mask) + continue; + /* vcpu open for airq of this ISC ? */ + if (!(ioint_mask & isc_mask)) + continue; + /* use this vcpu (for all ISCs in ioint_mask) */ + kick_mask |= ioint_mask; > + kick_vcpu[kicked++] = vcpu; + } + } + + if (vcpu && ~kick_mask & ipm)arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2987
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+ VM_EVENT(kvm, 4, "gib alert undeliverable isc mask 0x%02x", + ~kick_mask & ipm); + + for (vcpu_id = 0; vcpu_id < kicked; vcpu_id++) + kvm_s390_vcpu_wakeup(kick_vcpu[vcpu_id]); + + return (online_vcpus != 0) ? kicked : -ENODEV; +} + +static void __floating_airqs_kick (struct kvm *kvm)static inline ?
Quoted 10 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2994
Quoted 7 lines
arch/s390/kvm/interrupt.c +7
-0+{ + struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; + int online_vcpus, kicked; + u8 ipm_t0, ipm; + + /* Get IPM and return if clean, IAM has been restored. */ + ipm = get_ipm(kvm->arch.gisa, IRQ_FLAG_IAM);If we do not get an IPM here, it must have been stolen by the firmware for delivery to the guest.
Yes, a running SIE instance took it before we were able to. But is it still running now? It could have gone to WAIT before we see that the IPM is clean. Then it was restored already. Otherwise, it is still running and will go WAIT and then restore the IAM.
I will do some tests on this.
Quoted 85 lines
Then why restoring the IAM?
Or do I miss something?
Quoted 35 lines
arch/s390/kvm/interrupt.c +35
-0+ if (!ipm) + return; +retry: + ipm_t0 = ipm; + + /* Try to kick some vcpus in WAIT state. */ + kicked = __try_airqs_kick(kvm, ipm); + if (kicked < 0) + return; + + /* Get IPM and return if clean, IAM has been restored. */ + ipm = get_ipm(kvm->arch.gisa, IRQ_FLAG_IAM); + if (!ipm) + return; + + /* Start over, if new ISC bits are pending in IPM. */ + if ((ipm_t0 ^ ipm) & ~ipm_t0) + goto retry; + + /* + * Return as we just kicked at least one vcpu in WAIT state + * open for airqs. The IAM will be restored latest when one + * of them goes into WAIT or STOP state. + */ + if (kicked > 0) + return; + + /* +* No vcpu was kicked either because no vcpu was in WAIT state +* or none of the vcpus in WAIT state are open for airqs. +* Return immediately if no vcpus are in WAIT state. +* There are vcpus in RUN state. They will process the airqs +* if not closed for airqs as well. In that case the system will +* delay airqs until a vcpu decides to take airqs again. +*/ arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3039
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+ online_vcpus = atomic_read(&kvm->online_vcpus); + if (!bitmap_weight(fi->idle_mask, online_vcpus)) + return; + + /* + * None of the vcpus in WAIT state take airqs and we might + * have no running vcpus as at least one vcpu is in WAIT state + * and IPM is dirty. + */ + set_iam(kvm->arch.gisa, kvm->arch.iam);I do not understand why we need to set IAM here.
The interrupt will be delivered by the firmware as soon as the PSW or CR6 is changed by any vCPU.
...and if this does not happen we can not deliver the interrupt anyway.Quoted 30 lines
arch/s390/kvm/interrupt.c +30
-0+} + +#define NULL_GISA_ADDR 0x00000000UL +#define NONE_GISA_ADDR 0x00000001UL +#define GISA_ADDR_MASK 0xfffff000UL + +static void __maybe_unused process_gib_alert_list (void ) +{ + u32 final, next_alert, origin = 0UL; + struct kvm_s390_gisa *gisa; + struct kvm *kvm; + + do { + /* + * If the NONE_GISA_ADDR is still stored in the alert list + * origin, we will leave the outer loop. No further GISA has + * been added to the alert list by millicode while processing + * the current alert list. + */ + final = (origin & NONE_GISA_ADDR); + /* + * Cut off the alert list and store the NONE_GISA_ADDR in the + * alert list origin to avoid further GAL interruptions. + * A new alert list can be build up by millicode in parallel + * for guests not in the yet cut-off alert list. When in the + * final loop, store the NULL_GISA_ADDR instead. This will re- + * enable GAL interruptions on the host again. + */ + origin = xchg(&gib->alert_list_origin, + (!final) ? NONE_GISA_ADDR : NULL_GISA_ADDR);Quoted 8 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3075
Quoted 6 lines
arch/s390/kvm/interrupt.c +6
-0+ /* Loop through the just cut-off alert list. */ + while (origin & GISA_ADDR_MASK) { + gisa = (struct kvm_s390_gisa *)(u64)origin; + next_alert = gisa->next_alert; + /* Unlink the GISA from the alert list. */ + gisa->next_alert = origin;AFAIU this enable GISA interrupt for the guest...
Only together with the IAM being set what could happen if
__floating_airqs_kick()callsget_ipmand the IPM is clean already. :(Quoted 7 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3078
Quoted 4 lines
arch/s390/kvm/interrupt.c +3
-0+ kvm = container_of(gisa, struct sie_page2, gisa)->kvm; + /* Kick suitable vcpus */ + __floating_airqs_kick(kvm);...and here we kick a VCPU for the guest.
Quoted 7 lines
Logically I would do it in the otherway, first kicking the vCPU then enabling the GISA interruption again.
If the IPM bit is cleared by the firmware during delivering the interrupt to the guest before we enter
get_ipm()called by__floating_airqs_kick()we will set the IAM despite we have a running CPU handling the IRQ.
Quoted 9 lines
I will move the unlink below the kick that will assure
get_ipmwill never take the IAM restore path.Quoted 3 lines
In the worst case we can also set the IAM with the GISA in the alert list. Or we must accept that the firmware can deliver the IPM as soon as we reset the GISA next field.
See statement above.
I'm very confused by these comments, and especially by your apparent consensus.
Regards,
Halil
Re: [PATCH v5 13/15] KVM: s390: add function process_gib_alert_list()
In reply to Michael MuellerQuoted 5 lines · Michael Mueller
Quoted 2 lines · Pierre Morel
Quoted 1 line · Michael Mueller
This function processes the Gib Alert List (GAL). It is required
...snip...
Quoted 27 lines
+ struct kvm *kvm;
Quoted 26 lines
Quoted 25 lines
+
+ do {
+ /*
+ * If theNONE_GISA_ADDRis still stored in the alert list + * origin, we will leave the outer loop. No further GISA has + * been added to the alert list by millicode while processing + * the current alert list.
+ */
+ final = (origin &NONE_GISA_ADDR);
+ /*
+ * Cut off the alert list and store theNONE_GISA_ADDRin the + * alert list origin to avoid further GAL interruptions. + * A new alert list can be build up by millicode in parallel + * for guests not in the yet cut-off alert list. When in the + * final loop, store theNULL_GISA_ADDRinstead. This will re- + * enable GAL interruptions on the host again.
+ */
+ origin = xchg(&gib->alert_list_origin,
+ (!final) ?NONE_GISA_ADDR:NULL_GISA_ADDR); + /* Loop through the just cut-off alert list. */
+ while (origin &GISA_ADDR_MASK) {
+ gisa = (structkvm_s390_gisa*)(u64)origin;
+next_alert=gisa->next_alert;
+ /* Unlink the GISA from the alert list. */
+gisa->next_alert= origin;
Quoted 4 lines
Quoted 1 line
AFAIU this enable GISA interrupt for the guest...
Only together with the IAM being set what could happen if
__floating_airqs_kick()callsget_ipmand the IPM is clean already. :(
confused, AFAIK IAM is used to allow interrupt for the host not for the guest.
Quoted 9 lines
Quoted 9 lines
Quoted 5 lines
+ kvm =
container_of(gisa, struct sie_page2, gisa)->kvm; + /* Kick suitable vcpus */
+__floating_airqs_kick(kvm);...and here we kick a VCPU for the guest.
Logically I would do it in the otherway, first kicking the vCPU then enabling the GISA interruption again.
!! sorry to have introduce this confusion.
You did it in the right order.
I should have not send these comments after I gave my R-B
Quoted 4 lines
Quoted 2 lines
If the IPM bit is cleared by the firmware during delivering the interrupt to the guest before we enter
get_ipm()called by__floating_airqs_kick()we will set the IAM despite we have a running CPU handling the IRQ.I will move the unlink below the kick that will assure
get_ipmwill never take the IAM restore path.
!! Sorry, you were right.
We must re-enable interrupt before kicking the vcpu, as you did, or the vcpu could go to wait before it gets the interrupt.
Quoted 18 lines
Quoted 3 lines
In the worst case we can also set the IAM with the GISA in the alert list.
Or we must accept that the firmware can deliver the IPM as soon as we reset the GISA next field.See statement above.
Quoted 12 lines
Quoted 10 lines
+ origin =
next_alert;
+ }
+ } while (!final);(quoted diff) +2
-0+} +static void
nullify_gisa(struct kvm_s390_gisa *gisa){
memset(gisa, 0, sizeof(struct kvm_s390_gisa));I think that avoiding to restore the IAM during the call to
get_ipminside__floating_airqs_kick()would good.
I still think tis assumption is right: We should not set the IAM during the kick.
Quoted 4 lines
Quoted 4 lines
If you agree, with that:
Reviewed-by Pierre Morel<[email protected]>
Still OK with my R-B, as long as w do not set IAM during the kicking.
Regards,
Pierre
Signature · 2 lines
Pierre Morel Linux/KVM/QEMU in Böblingen - Germany
Re: [PATCH v5 13/15] KVM: s390: add function process_gib_alert_list()
In reply to Pierre MorelQuoted 15 lines · Pierre Morel
Quoted 14 lines · Michael Mueller
This function processes the Gib Alert List (GAL). It is required to run when either a gib alert interruption has been received or a gisa that is in the alert list is cleared or dropped.
The GAL is build up by millicode, when the respective ISC bit is set in the Interruption Alert Mask (IAM) and an interruption of that class is observed.
Signed-off-by Michael Mueller <[email protected]>---
arch/s390/kvm/interrupt.c| 140
++++++++++++++++++++++++++++++++++++++++++++++1 file changed, 140 insertions(+)
Quoted 11 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu · line 2944
Quoted 9 lines
arch/s390/kvm/interrupt.c +0
-0@@ -2941,6 +2941,146 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) return n; } +static int __try_airqs_kick(struct kvm *kvm, u8 ipm)static inline ?
will add
Quoted 37 lines
Quoted 37 lines
arch/s390/kvm/interrupt.c +35
-0+{ + struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; + struct kvm_vcpu *vcpu = NULL, *kick_vcpu[MAX_ISC + 1]; + int online_vcpus = atomic_read(&kvm->online_vcpus); + u8 ioint_mask, isc_mask, kick_mask = 0x00; + int vcpu_id, kicked = 0; + + /* Loop over vcpus in WAIT state. */ + for (vcpu_id = find_first_bit(fi->idle_mask, online_vcpus); + /* Until all pending ISCs have a vcpu open for airqs. */ + (~kick_mask & ipm) && vcpu_id < online_vcpus; + vcpu_id = find_next_bit(fi->idle_mask, online_vcpus, vcpu_id)) { + vcpu = kvm_get_vcpu(kvm, vcpu_id); + if (psw_ioint_disabled(vcpu)) + continue; + ioint_mask = (u8)(vcpu->arch.sie_block->gcr[6] >> 24); + for (isc_mask = 0x80; isc_mask; isc_mask >>= 1) { + /* ISC pending in IPM ? */ + if (!(ipm & isc_mask)) + continue; + /* vcpu for this ISC already found ? */ + if (kick_mask & isc_mask) + continue; + /* vcpu open for airq of this ISC ? */ + if (!(ioint_mask & isc_mask)) + continue; + /* use this vcpu (for all ISCs in ioint_mask) */ + kick_mask |= ioint_mask; > + kick_vcpu[kicked++] = vcpu; + } + } + + if (vcpu && ~kick_mask & ipm) + VM_EVENT(kvm, 4, "gib alert undeliverable isc mask 0x%02x", + ~kick_mask & ipm); +
Quoted 8 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2987
Quoted 7 lines
arch/s390/kvm/interrupt.c +7
-0+ for (vcpu_id = 0; vcpu_id < kicked; vcpu_id++) + kvm_s390_vcpu_wakeup(kick_vcpu[vcpu_id]); + + return (online_vcpus != 0) ? kicked : -ENODEV; +} + +static void __floating_airqs_kick (struct kvm *kvm)static inline ?
and here as well
Quoted 132 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2994
Quoted 8 lines
arch/s390/kvm/interrupt.c +7
-0+{ + struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; + int online_vcpus, kicked; + u8 ipm_t0, ipm; + + /* Get IPM and return if clean, IAM has been restored. */ + ipm = get_ipm(kvm->arch.gisa, IRQ_FLAG_IAM);If we do not get an IPM here, it must have been stolen by the firmware for delivery to the guest. Then why restoring the IAM?
Or do I miss something?
Quoted 35 lines
arch/s390/kvm/interrupt.c +35
-0+ if (!ipm) + return; +retry: + ipm_t0 = ipm; + + /* Try to kick some vcpus in WAIT state. */ + kicked = __try_airqs_kick(kvm, ipm); + if (kicked < 0) + return; + + /* Get IPM and return if clean, IAM has been restored. */ + ipm = get_ipm(kvm->arch.gisa, IRQ_FLAG_IAM); + if (!ipm) + return; + + /* Start over, if new ISC bits are pending in IPM. */ + if ((ipm_t0 ^ ipm) & ~ipm_t0) + goto retry; + + /* + * Return as we just kicked at least one vcpu in WAIT state + * open for airqs. The IAM will be restored latest when one + * of them goes into WAIT or STOP state. + */ + if (kicked > 0) + return; + + /* +* No vcpu was kicked either because no vcpu was in WAIT state +* or none of the vcpus in WAIT state are open for airqs. +* Return immediately if no vcpus are in WAIT state. +* There are vcpus in RUN state. They will process the airqs +* if not closed for airqs as well. In that case the system will +* delay airqs until a vcpu decides to take airqs again. +*/ arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3039
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+ online_vcpus = atomic_read(&kvm->online_vcpus); + if (!bitmap_weight(fi->idle_mask, online_vcpus)) + return; + + /* + * None of the vcpus in WAIT state take airqs and we might + * have no running vcpus as at least one vcpu is in WAIT state + * and IPM is dirty. + */ + set_iam(kvm->arch.gisa, kvm->arch.iam);I do not understand why we need to set IAM here.
The interrupt will be delivered by the firmware as soon as the PSW or CR6 is changed by any vCPU. ...and if this does not happen we can not deliver the interrupt anyway.Quoted 26 lines
arch/s390/kvm/interrupt.c +26
-0+} + +#define NULL_GISA_ADDR 0x00000000UL +#define NONE_GISA_ADDR 0x00000001UL +#define GISA_ADDR_MASK 0xfffff000UL + +static void __maybe_unused process_gib_alert_list (void ) +{ + u32 final, next_alert, origin = 0UL; + struct kvm_s390_gisa *gisa; + struct kvm *kvm; + + do { + /* + * If the NONE_GISA_ADDR is still stored in the alert list + * origin, we will leave the outer loop. No further GISA has + * been added to the alert list by millicode while processing + * the current alert list. + */ + final = (origin & NONE_GISA_ADDR); + /* + * Cut off the alert list and store the NONE_GISA_ADDR in the + * alert list origin to avoid further GAL interruptions. + * A new alert list can be build up by millicode in parallel + * for guests not in the yet cut-off alert list. When in the + * final loop, store the NULL_GISA_ADDR instead. This will re-arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3075
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+* enable GAL interruptions on the host again. +*/ + origin = xchg(&gib->alert_list_origin, + (!final) ? NONE_GISA_ADDR : NULL_GISA_ADDR); + /* Loop through the just cut-off alert list. */ + while (origin & GISA_ADDR_MASK) { + gisa = (struct kvm_s390_gisa *)(u64)origin; + next_alert = gisa->next_alert; + /* Unlink the GISA from the alert list. */ + gisa->next_alert = origin;AFAIU this enable GISA interrupt for the guest...
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3078
Quoted 3 lines
arch/s390/kvm/interrupt.c +3
-0+ kvm = container_of(gisa, struct sie_page2, gisa)->kvm; + /* Kick suitable vcpus */ + __floating_airqs_kick(kvm);...and here we kick a VCPU for the guest.
Logically I would do it in the otherway, first kicking the vCPU then enabling the GISA interruption again.
If the IPM bit is cleared by the firmware during delivering the interrupt to the guest before we enter
get_ipm()called by__floating_airqs_kick()we will set the IAM despite we have a running CPU handling the IRQ. In the worst case we can also set the IAM with the GISA in the alert list.
Or we must accept that the firmware can deliver the IPM as soon as we reset the GISA next field.Quoted 9 lines
arch/s390/kvm/interrupt.c +5
-0+ origin = next_alert; + } + } while (!final); +} + static void nullify_gisa(struct kvm_s390_gisa *gisa){
memset(gisa, 0, sizeof(struct kvm_s390_gisa));I think that avoiding to restore the IAM during the call to
get_ipminside__floating_airqs_kick()would good.If you agree, with that:
Reviewed-by Pierre Morel<[email protected]>
Signature · 8 lines
Mit freundlichen Grüßen / Kind regards Michael Müller IBM Deutschland Research & Development GmbH Vorsitzender des Aufsichtsrats: Matthias Hartmann Geschäftsführung: Dirk Wittkopp Sitz der Gesellschaft: Böblingen Registergericht: Amtsgericht Stuttgart, HRB 243294
Re: [PATCH v5 13/15] KVM: s390: add function process_gib_alert_list()
Quoted 17 lines · Michael Mueller
Quoted 15 lines · Pierre Morel
Quoted 14 lines · Michael Mueller
This function processes the Gib Alert List (GAL). It is required to run when either a gib alert interruption has been received or a gisa that is in the alert list is cleared or dropped.
The GAL is build up by millicode, when the respective ISC bit is set in the Interruption Alert Mask (IAM) and an interruption of that class is observed.
Signed-off-by Michael Mueller <[email protected]>---
arch/s390/kvm/interrupt.c| 140
++++++++++++++++++++++++++++++++++++++++++++++1 file changed, 140 insertions(+)
Quoted 11 lines
Quoted 11 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu · line 2944
Quoted 9 lines
arch/s390/kvm/interrupt.c +0
-0@@ -2941,6 +2941,146 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) return n; } +static int __try_airqs_kick(struct kvm *kvm, u8 ipm)static inline ?
Why?
In general it is a good idea to leave it up to the compiler to decide if it is worth to inline a function or not, unless you have a good reason to force inlining.
Re: [PATCH v5 13/15] KVM: s390: add function process_gib_alert_list()
In reply to Michael MuellerQuoted 41 lines · On Wed, 19 Dec 2018 20:17:54 +0100 Michael Mueller <[email protected]> wrote:
This function processes the Gib Alert List (GAL). It is required to run when either a gib alert interruption has been received or a gisa that is in the alert list is cleared or dropped.
The GAL is build up by millicode, when the respective ISC bit is set in the Interruption Alert Mask (IAM) and an interruption of that class is observed.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 140 insertions(+)arch/s390/kvm/interrupt.c +20
-0@@ -2941,6 +2941,146 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len)return n; } +static int __try_airqs_kick (struct kvm *kvm,u8 ipm) +{ +struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; +struct kvm_vcpu *vcpu =NULL , *kick_vcpu[MAX_ISC +1 ]; +int online_vcpus =atomic_read (&kvm->online_vcpus); +u8 ioint_mask, isc_mask, kick_mask =0x00 ; +int vcpu_id, kicked =0 ; + +/* Loop over vcpus in WAIT state. */ +for (vcpu_id =find_first_bit (fi->idle_mask, online_vcpus); +/* Until all pending ISCs have a vcpu open for airqs. */ + (~kick_mask & ipm) && vcpu_id < online_vcpus; + vcpu_id =find_next_bit (fi->idle_mask, online_vcpus, vcpu_id)) { + vcpu =kvm_get_vcpu (kvm, vcpu_id); +if (psw_ioint_disabled (vcpu)) +continue ; + ioint_mask = (u8)(vcpu->arch.sie_block->gcr[6 ] >>24 ); +for (isc_mask =0x80 ; isc_mask; isc_mask >>=1 ) { +/* ISC pending in IPM ? */ +if (!(ipm & isc_mask))
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2973
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+continue ; +/* vcpu for this ISC already found ? */ +if (kick_mask & isc_mask) +continue ; +/* vcpu open for airq of this ISC ? */ +if (!(ioint_mask & isc_mask)) +continue ; +/* use this vcpu (for all ISCs in ioint_mask) */ + kick_mask |= ioint_mask; + kick_vcpu[kicked++] = vcpu;
Assuming that the vcpu can/will take all ISCs it's currently open for does not seem right. We kind of rely on this assumption here, or?
Quoted 31 lines
arch/s390/kvm/interrupt.c +30
-0+ } + } + +if (vcpu && ~kick_mask & ipm) + VM_EVENT(kvm, 4, "gib alert undeliverable isc mask 0x%02x", + ~kick_mask & ipm); + +for (vcpu_id =0 ; vcpu_id < kicked; vcpu_id++) +kvm_s390_vcpu_wakeup (kick_vcpu[vcpu_id]); + +return (online_vcpus !=0 ) ? kicked : -ENODEV ; +} + +static void __floating_airqs_kick (struct kvm *kvm) +{ +struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; +int online_vcpus, kicked; +u8 ipm_t0, ipm; + +/* Get IPM and return if clean, IAM has been restored. */ + ipm =get_ipm (kvm->arch.gisa,IRQ_FLAG_IAM ); +if (!ipm) +return ; +retry: + ipm_t0 = ipm; + +/* Try to kick some vcpus in WAIT state. */ + kicked =__try_airqs_kick (kvm, ipm); +if (kicked <0 ) +return ;
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3013
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+ +/* Get IPM and return if clean, IAM has been restored. */ + ipm =get_ipm (kvm->arch.gisa,IRQ_FLAG_IAM ); +if (!ipm) +return ; + +/* Start over, if new ISC bits are pending in IPM. */ +if ((ipm_t0 ^ ipm) & ~ipm_t0) + goto retry; +
<MARK A>
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3020
Quoted 7 lines
arch/s390/kvm/interrupt.c +7
-0+/* +* Return as we just kicked at least one vcpu in WAIT state +* open for airqs. The IAM will be restored latest when one +* of them goes into WAIT or STOP state. +*/ +if (kicked >0 ) +return ;
</MARK A>
Quoted 48 lines
arch/s390/kvm/interrupt.c +48
-0+ +/* +* No vcpu was kicked either because no vcpu was in WAIT state +* or none of the vcpus in WAIT state are open for airqs. +* Return immediately if no vcpus are in WAIT state. +* There are vcpus in RUN state. They will process the airqs +* if not closed for airqs as well. In that case the system will +* delay airqs until a vcpu decides to take airqs again. +*/ + online_vcpus =atomic_read (&kvm->online_vcpus); +if (!bitmap_weight (fi->idle_mask, online_vcpus)) +return ; + +/* +* None of the vcpus in WAIT state take airqs and we might +* have no running vcpus as at least one vcpu is in WAIT state +* and IPM is dirty. +*/ +set_iam (kvm->arch.gisa, kvm->arch.iam); +} + +#define NULL_GISA_ADDR 0x00000000UL +#define NONE_GISA_ADDR 0x00000001UL +#define GISA_ADDR_MASK 0xfffff000UL + +static void __maybe_unused process_gib_alert_list(void) +{ + u32 final, next_alert, origin = 0UL; + struct kvm_s390_gisa *gisa; + struct kvm *kvm; + + do { + /* + * If the NONE_GISA_ADDR is still stored in the alert list + * origin, we will leave the outer loop. No further GISA has + * been added to the alert list by millicode while processing + * the current alert list. + */ + final = (origin & NONE_GISA_ADDR); + /* + * Cut off the alert list and store the NONE_GISA_ADDR in the + * alert list origin to avoid further GAL interruptions. + * A new alert list can be build up by millicode in parallel + * for guests not in the yet cut-off alert list. When in the + * final loop, store the NULL_GISA_ADDR instead. This will re- + * enable GAL interruptions on the host again. + */ + origin = xchg(&gib->alert_list_origin,
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3078
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+ (!final) ? NONE_GISA_ADDR : NULL_GISA_ADDR); +/* Loop through the just cut-off alert list. */ +while (origin &GISA_ADDR_MASK ) { + gisa = (struct kvm_s390_gisa *)(u64 )origin; + next_alert = gisa->next_alert; +/* Unlink the GISA from the alert list. */ + gisa->next_alert = origin; + kvm = container_of(gisa, struct sie_page2, gisa)->kvm; +/* Kick suitable vcpus */ +__floating_airqs_kick (kvm);
We may finish handling the alerted gisa with iam not set e.g. via some vcpus kicked but ipm still dirty and some vcpus still in wait, or?
Quoted 1 line
From the comments it seems we speculate on being in a safe state, as
these are supposed to return to wait or stop soon-ish, and we will set iam then (See <MARK A>). I don't quite understand.
According to my current understanding we might end up loosing initiative in this scenario. Or am I wrong?
Regards,
Halil
Quoted 8 lines
arch/s390/kvm/interrupt.c +5
-0+ origin = next_alert; + } + }while (!final); +} +static void nullify_gisa (struct kvm_s390_gisa *gisa) {memset (gisa,0 ,sizeof (struct kvm_s390_gisa ));
Re: [PATCH v5 13/15] KVM: s390: add function process_gib_alert_list()
Quoted 49 lines · Halil Pasic
Quoted 46 lines · On Wed, 19 Dec 2018 20:17:54 +0100 Michael Mueller <[email protected]> wrote:
This function processes the Gib Alert List (GAL). It is required to run when either a gib alert interruption has been received or a gisa that is in the alert list is cleared or dropped.
The GAL is build up by millicode, when the respective ISC bit is set in the Interruption Alert Mask (IAM) and an interruption of that class is observed.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 140 insertions(+)arch/s390/kvm/interrupt.c +25
-0@@ -2941,6 +2941,146 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len)return n; } +static int __try_airqs_kick (struct kvm *kvm,u8 ipm) +{ +struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; +struct kvm_vcpu *vcpu =NULL , *kick_vcpu[MAX_ISC +1 ]; +int online_vcpus =atomic_read (&kvm->online_vcpus); +u8 ioint_mask, isc_mask, kick_mask =0x00 ; +int vcpu_id, kicked =0 ; + +/* Loop over vcpus in WAIT state. */ +for (vcpu_id =find_first_bit (fi->idle_mask, online_vcpus); +/* Until all pending ISCs have a vcpu open for airqs. */ + (~kick_mask & ipm) && vcpu_id < online_vcpus; + vcpu_id =find_next_bit (fi->idle_mask, online_vcpus, vcpu_id)) { + vcpu =kvm_get_vcpu (kvm, vcpu_id); +if (psw_ioint_disabled (vcpu)) +continue ; + ioint_mask = (u8)(vcpu->arch.sie_block->gcr[6 ] >>24 ); +for (isc_mask =0x80 ; isc_mask; isc_mask >>=1 ) { +/* ISC pending in IPM ? */ +if (!(ipm & isc_mask)) +continue ; +/* vcpu for this ISC already found ? */ +if (kick_mask & isc_mask) +continue ; +/* vcpu open for airq of this ISC ? */
Quoted 8 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2973
Quoted 5 lines
arch/s390/kvm/interrupt.c +5
-0+if (!(ioint_mask & isc_mask)) +continue ; +/* use this vcpu (for all ISCs in ioint_mask) */ + kick_mask |= ioint_mask; + kick_vcpu[kicked++] = vcpu;Assuming that the vcpu can/will take all ISCs it's currently open for does not seem right. We kind of rely on this assumption here, or?
My latest version of this routine already follows a different strategy. It looks for a horizontal distribution of pending ISCs among idle vcpus.
Quoted 109 lines
Quoted 32 lines
arch/s390/kvm/interrupt.c +30
-0+ } + } + +if (vcpu && ~kick_mask & ipm) + VM_EVENT(kvm, 4, "gib alert undeliverable isc mask 0x%02x", + ~kick_mask & ipm); + +for (vcpu_id =0 ; vcpu_id < kicked; vcpu_id++) +kvm_s390_vcpu_wakeup (kick_vcpu[vcpu_id]); + +return (online_vcpus !=0 ) ? kicked : -ENODEV ; +} + +static void __floating_airqs_kick (struct kvm *kvm) +{ +struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; +int online_vcpus, kicked; +u8 ipm_t0, ipm; + +/* Get IPM and return if clean, IAM has been restored. */ + ipm =get_ipm (kvm->arch.gisa,IRQ_FLAG_IAM ); +if (!ipm) +return ; +retry: + ipm_t0 = ipm; + +/* Try to kick some vcpus in WAIT state. */ + kicked =__try_airqs_kick (kvm, ipm); +if (kicked <0 ) +return ;arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3013
Quoted 10 lines
arch/s390/kvm/interrupt.c +10
-0+ +/* Get IPM and return if clean, IAM has been restored. */ + ipm =get_ipm (kvm->arch.gisa,IRQ_FLAG_IAM ); +if (!ipm) +return ; + +/* Start over, if new ISC bits are pending in IPM. */ +if ((ipm_t0 ^ ipm) & ~ipm_t0) + goto retry; +<MARK A>
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3020
Quoted 7 lines
arch/s390/kvm/interrupt.c +7
-0+/* +* Return as we just kicked at least one vcpu in WAIT state +* open for airqs. The IAM will be restored latest when one +* of them goes into WAIT or STOP state. +*/ +if (kicked >0 ) +return ;</MARK A>
Quoted 54 lines
arch/s390/kvm/interrupt.c +54
-0+ +/* +* No vcpu was kicked either because no vcpu was in WAIT state +* or none of the vcpus in WAIT state are open for airqs. +* Return immediately if no vcpus are in WAIT state. +* There are vcpus in RUN state. They will process the airqs +* if not closed for airqs as well. In that case the system will +* delay airqs until a vcpu decides to take airqs again. +*/ + online_vcpus =atomic_read (&kvm->online_vcpus); +if (!bitmap_weight (fi->idle_mask, online_vcpus)) +return ; + +/* +* None of the vcpus in WAIT state take airqs and we might +* have no running vcpus as at least one vcpu is in WAIT state +* and IPM is dirty. +*/ +set_iam (kvm->arch.gisa, kvm->arch.iam); +} + +#define NULL_GISA_ADDR 0x00000000UL +#define NONE_GISA_ADDR 0x00000001UL +#define GISA_ADDR_MASK 0xfffff000UL + +static void __maybe_unused process_gib_alert_list (void ) +{ +u32 final, next_alert, origin =0UL ; +struct kvm_s390_gisa *gisa; +struct kvm *kvm; + + do { + /* + * If the NONE_GISA_ADDR is still stored in the alert list + * origin, we will leave the outer loop. No further GISA has + * been added to the alert list by millicode while processing + * the current alert list. + */ + final = (origin & NONE_GISA_ADDR); + /* + * Cut off the alert list and store the NONE_GISA_ADDR in the + * alert list origin to avoid further GAL interruptions. + * A new alert list can be build up by millicode in parallel + * for guests not in the yet cut-off alert list. When in the + * final loop, store the NULL_GISA_ADDR instead. This will re- + * enable GAL interruptions on the host again. + */ + origin = xchg(&gib->alert_list_origin, + (!final) ? NONE_GISA_ADDR : NULL_GISA_ADDR); + /* Loop through the just cut-off alert list. */ + while (origin & GISA_ADDR_MASK) { + gisa = (struct kvm_s390_gisa *)(u64)origin; + next_alert = gisa->next_alert; + /* Unlink the GISA from the alert list. */
Quoted 8 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3078
Quoted 4 lines
arch/s390/kvm/interrupt.c +4
-0+ gisa->next_alert = origin; + kvm = container_of(gisa, struct sie_page2, gisa)->kvm; +/* Kick suitable vcpus */ +__floating_airqs_kick (kvm);We may finish handling the alerted gisa with iam not set e.g. via some vcpus kicked but ipm still dirty and some vcpus still in wait, or?
My above mentioned change to the routine identifying the vcpus to kick will select one vcpu for each ISC pending if possible (depends on the number of idle vcpus and their respective interruption masks and the pending ISCs).
That does not exclude the principle scenario that maybe only one vcpu is kicked and multiple ISCs are pending (ipm still dirty) although have never observed this with a Linux guest.
What I was trying to avoid was a kind of busy loop running in addition to the kicked vcpus monitoring the IPM state for resource utilization reasons.
Quoted 4 lines
From the comments it seems we speculate on being in a safe state, as
these are supposed to return to wait or stop soon-ish, and we will set iam then (See <MARK A>). I don't quite understand.
Yes, the next vcpu going idle shall restore the IAM or process the
top ISC pending if the iomask (GCR) allows. vcpus are not allowed to go in disabled wait (IO int disabled by PSW). If all vcpus always (for some time) mask a specific ISC the guest does not want to get interrupted for that ISC but will as soon a running vcpu will open the mask again.
Quoted 3 lines
According to my current understanding we might end up loosing initiative in this scenario. Or am I wrong?
I currently don't have proof for you being wrong but have not observed the situation yet.
Quoted 4 lines
Regards,
Halil
Quoted 9 lines
Quoted 8 lines
arch/s390/kvm/interrupt.c +5
-0+ origin = next_alert; + } + }while (!final); +} +static void nullify_gisa (struct kvm_s390_gisa *gisa) {memset(gisa, 0, sizeof(struct kvm_s390_gisa));
-
Re: [PATCH v5 13/15] KVM: s390: add function process_gib_alert_list()
Quoted 9 lines · On Tue, 8 Jan 2019 16:21:17 +0100 Michael Mueller <[email protected]> wrote:
Quoted 8 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 3078
Quoted 4 lines
arch/s390/kvm/interrupt.c +4
-0+ gisa->next_alert = origin; + kvm = container_of(gisa, struct sie_page2, gisa)->kvm; +/* Kick suitable vcpus */ +__floating_airqs_kick (kvm);We may finish handling the alerted gisa with iam not set e.g. via some vcpus kicked but ipm still dirty and some vcpus still in wait, or?
Quoted 9 lines
My above mentioned change to the routine identifying the vcpus to kick will select one vcpu for each ISC pending if possible (depends on the number of idle vcpus and their respective interruption masks and the pending ISCs).
That does not exclude the principle scenario that maybe only one vcpu is kicked and multiple ISCs are pending (ipm still dirty) although have never observed this with a Linux guest.
IMHO we have to differentiate between the general case and between what can happen with current or historical Linux guests.
Regarding Linux guests, I'm under the impression each version was quite there. I says so, also because I have a reasonable amount of confidence in your testing.
Quoted 4 lines
What I was trying to avoid was a kind of busy loop running in addition to the kicked vcpus monitoring the IPM state for resource utilization reasons.
Got it. I think we need a clean switch-over (between our code makes sure the no pending interrupts are going to get stalled in spite of waiting vcpus that could take them, and between we are good now and if we are not good any more we will get an alert) nevertheless.
Quoted 6 lines
Quoted 4 lines
From the comments it seems we speculate on being in a safe state, as
these are supposed to return to wait or stop soon-ish, and we will set iam then (See <MARK A>). I don't quite understand.
Quoted 7 lines
Yes, the next vcpu going idle shall restore the IAM or process the top ISC pending if the iomask (GCR) allows. vcpus are not allowed to go in disabled wait (IO int disabled by PSW). If all vcpus always (for some time) mask a specific ISC the guest does not want to get interrupted for that ISC but will as soon a running vcpu will open the mask again.
My understanding on the guarantees we can provide based on the fact that we kicked some vcpus is still lacking. Maybe let us discuss this offline.
Quoted 6 lines
Quoted 3 lines
According to my current understanding we might end up loosing initiative in this scenario. Or am I wrong?
I currently don't have proof for you being wrong but have not observed the situation yet.
See above. Proving that no irq's can get substantially delayed needlessly (where needlessly means, there is a vcpu in wait state that could take the irq) would be a proof enough that I'm wrong. Let's make this discussion more efficient by utilizing co-location to cut down the RTT.
Regards,
Halil
Re: [PATCH v5 13/15] KVM: s390: add function process_gib_alert_list()
In reply to Michael MuellerQuoted 54 lines · Michael Mueller
Quoted 51 lines · Halil Pasic
Quoted 48 lines · On Wed, 19 Dec 2018 20:17:54 +0100 Michael Mueller <[email protected]> wrote:
This function processes the Gib Alert List (GAL). It is required to run when either a gib alert interruption has been received or a gisa that is in the alert list is cleared or dropped.
The GAL is build up by millicode, when the respective ISC bit is set in the Interruption Alert Mask (IAM) and an interruption of that class is observed.
Signed-off-by Michael Mueller <[email protected]>---
arch/s390/kvm/interrupt.c| 140
++++++++++++++++++++++++++++++++++++++++++++++1 file changed, 140 insertions(+)arch/s390/kvm/interrupt.c +25
-0@@ -2941,6 +2941,146 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) return n; } +static int __try_airqs_kick (struct kvm *kvm,u8 ipm) +{ + struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; + struct kvm_vcpu *vcpu = NULL, *kick_vcpu[MAX_ISC + 1]; + int online_vcpus = atomic_read(&kvm->online_vcpus); + u8 ioint_mask, isc_mask, kick_mask = 0x00; + int vcpu_id, kicked = 0; + + /* Loop over vcpus in WAIT state. */ + for (vcpu_id = find_first_bit(fi->idle_mask, online_vcpus); + /* Until all pending ISCs have a vcpu open for airqs. */ + (~kick_mask & ipm) && vcpu_id < online_vcpus; + vcpu_id = find_next_bit(fi->idle_mask, online_vcpus, vcpu_id)) { + vcpu = kvm_get_vcpu(kvm, vcpu_id); + if (psw_ioint_disabled(vcpu)) + continue; + ioint_mask = (u8)(vcpu->arch.sie_block->gcr[6] >> 24); + for (isc_mask = 0x80; isc_mask; isc_mask >>= 1) { + /* ISC pending in IPM ? */ + if (!(ipm & isc_mask)) + continue; + /* vcpu for this ISC already found ? */ + if (kick_mask & isc_mask) + continue; + /* vcpu open for airq of this ISC ? */
Quoted 8 lines
Quoted 8 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2973
Quoted 5 lines
arch/s390/kvm/interrupt.c +5
-0+ if (!(ioint_mask & isc_mask)) + continue; + /* use this vcpu (for all ISCs in ioint_mask) */ + kick_mask |= ioint_mask; + kick_vcpu[kicked++] = vcpu;Assuming that the vcpu can/will take all ISCs it's currently open for does not seem right. We kind of rely on this assumption here, or?
why does it not seem right?
Quoted 4 lines
My latest version of this routine already follows a different strategy. It looks for a horizontal distribution of pending ISCs among idle vcpus.
May be you should separate the GAL IRQ handling and the algorithm of the vCPU to kick in different patches to ease the review.
Regards,
Pierre
Signature · 2 lines
Pierre Morel Linux/KVM/QEMU in Böblingen - Germany
Re: [PATCH v5 13/15] KVM: s390: add function process_gib_alert_list()
Quoted 64 lines · On Wed, 9 Jan 2019 13:14:17 +0100 Pierre Morel <[email protected]> wrote:
Quoted 54 lines · Michael Mueller
Quoted 51 lines · Halil Pasic
Quoted 48 lines · On Wed, 19 Dec 2018 20:17:54 +0100 Michael Mueller <[email protected]> wrote:
This function processes the Gib Alert List (GAL). It is required to run when either a gib alert interruption has been received or a gisa that is in the alert list is cleared or dropped.
The GAL is build up by millicode, when the respective ISC bit is set in the Interruption Alert Mask (IAM) and an interruption of that class is observed.
Signed-off-by Michael Mueller <[email protected]>---
arch/s390/kvm/interrupt.c| 140
++++++++++++++++++++++++++++++++++++++++++++++1 file changed, 140 insertions(+)arch/s390/kvm/interrupt.c +25
-0@@ -2941,6 +2941,146 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) return n; } +static int __try_airqs_kick (struct kvm *kvm,u8 ipm) +{ + struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; + struct kvm_vcpu *vcpu = NULL, *kick_vcpu[MAX_ISC + 1]; + int online_vcpus = atomic_read(&kvm->online_vcpus); + u8 ioint_mask, isc_mask, kick_mask = 0x00; + int vcpu_id, kicked = 0; + + /* Loop over vcpus in WAIT state. */ + for (vcpu_id = find_first_bit(fi->idle_mask, online_vcpus); + /* Until all pending ISCs have a vcpu open for airqs. */ + (~kick_mask & ipm) && vcpu_id < online_vcpus; + vcpu_id = find_next_bit(fi->idle_mask, online_vcpus, vcpu_id)) { + vcpu = kvm_get_vcpu(kvm, vcpu_id); + if (psw_ioint_disabled(vcpu)) + continue; + ioint_mask = (u8)(vcpu->arch.sie_block->gcr[6] >> 24); + for (isc_mask = 0x80; isc_mask; isc_mask >>= 1) { + /* ISC pending in IPM ? */ + if (!(ipm & isc_mask)) + continue; + /* vcpu for this ISC already found ? */ + if (kick_mask & isc_mask) + continue; + /* vcpu open for airq of this ISC ? */Quoted 8 lines
Quoted 8 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2973
Quoted 5 lines
arch/s390/kvm/interrupt.c +5
-0+ if (!(ioint_mask & isc_mask)) + continue; + /* use this vcpu (for all ISCs in ioint_mask) */ + kick_mask |= ioint_mask; + kick_vcpu[kicked++] = vcpu;Assuming that the vcpu can/will take all ISCs it's currently open for does not seem right. We kind of rely on this assumption here, or?
Quoted 2 lines
why does it not seem right?
When an interrupt is delivered a psw-swap takes place. The new-psw may fence IO interrupts. Thus for example if we have the vcpu open for all ISCs and 0, 1 and 2 pending, we may end up only delivering 0, if the psw-swap corresponding to delivering 0 closes the vcpu for IO interrupts. After guest has control, we don't have control over the rest of the story.
Quoted 9 lines
Quoted 4 lines
My latest version of this routine already follows a different strategy. It looks for a horizontal distribution of pending ISCs among idle vcpus.
May be you should separate the GAL IRQ handling and the algorithm of the vCPU to kick in different patches to ease the review.
No strong opinion here. I found it convenient to have the most of the logic in one patch/email.
Regards,
Halil
Re: [PATCH v5 13/15] KVM: s390: add function process_gib_alert_list()
Quoted 70 lines · Halil Pasic
Quoted 64 lines · On Wed, 9 Jan 2019 13:14:17 +0100 Pierre Morel <[email protected]> wrote:
Quoted 54 lines · Michael Mueller
Quoted 51 lines · Halil Pasic
Quoted 48 lines · On Wed, 19 Dec 2018 20:17:54 +0100 Michael Mueller <[email protected]> wrote:
This function processes the Gib Alert List (GAL). It is required to run when either a gib alert interruption has been received or a gisa that is in the alert list is cleared or dropped.
The GAL is build up by millicode, when the respective ISC bit is set in the Interruption Alert Mask (IAM) and an interruption of that class is observed.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 140 insertions(+)arch/s390/kvm/interrupt.c +25
-0@@ -2941,6 +2941,146 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) return n; } +static int __try_airqs_kick (struct kvm *kvm,u8 ipm) +{ + struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; + struct kvm_vcpu *vcpu = NULL, *kick_vcpu[MAX_ISC + 1]; + int online_vcpus = atomic_read(&kvm->online_vcpus); + u8 ioint_mask, isc_mask, kick_mask = 0x00; + int vcpu_id, kicked = 0; + + /* Loop over vcpus in WAIT state. */ + for (vcpu_id = find_first_bit(fi->idle_mask, online_vcpus); + /* Until all pending ISCs have a vcpu open for airqs. */ + (~kick_mask & ipm) && vcpu_id < online_vcpus; + vcpu_id = find_next_bit(fi->idle_mask, online_vcpus, vcpu_id)) { + vcpu = kvm_get_vcpu(kvm, vcpu_id); + if (psw_ioint_disabled(vcpu)) + continue; + ioint_mask = (u8)(vcpu->arch.sie_block->gcr[6] >> 24); + for (isc_mask = 0x80; isc_mask; isc_mask >>= 1) { + /* ISC pending in IPM ? */ + if (!(ipm & isc_mask)) + continue; + /* vcpu for this ISC already found ? */ + if (kick_mask & isc_mask) + continue; + /* vcpu open for airq of this ISC ? */Quoted 8 lines
Quoted 8 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2973
Quoted 5 lines
arch/s390/kvm/interrupt.c +5
-0+ if (!(ioint_mask & isc_mask)) + continue; + /* use this vcpu (for all ISCs in ioint_mask) */ + kick_mask |= ioint_mask; + kick_vcpu[kicked++] = vcpu;Assuming that the vcpu can/will take all ISCs it's currently open for does not seem right. We kind of rely on this assumption here, or?
Quoted 2 lines
why does it not seem right?
Quoted 6 lines
When an interrupt is delivered a psw-swap takes place. The new-psw may fence IO interrupts. Thus for example if we have the vcpu open for all ISCs and 0, 1 and 2 pending, we may end up only delivering 0, if the psw-swap corresponding to delivering 0 closes the vcpu for IO interrupts. After guest has control, we don't have control over the rest of the story.
OK I think I understand your concern, waking up a single waiting vCPU per ISC is not enough. We must wake all vCPU in wait state having at least one matching ISC bit.
Regards,
Pierre
Signature · 2 lines
Pierre Morel Linux/KVM/QEMU in Böblingen - Germany
Re: [PATCH v5 13/15] KVM: s390: add function process_gib_alert_list()
Quoted 78 lines · On Wed, 9 Jan 2019 15:49:56 +0100 Pierre Morel <[email protected]> wrote:
Quoted 70 lines · Halil Pasic
Quoted 64 lines · On Wed, 9 Jan 2019 13:14:17 +0100 Pierre Morel <[email protected]> wrote:
Quoted 54 lines · Michael Mueller
Quoted 51 lines · Halil Pasic
Quoted 48 lines · On Wed, 19 Dec 2018 20:17:54 +0100 Michael Mueller <[email protected]> wrote:
This function processes the Gib Alert List (GAL). It is required to run when either a gib alert interruption has been received or a gisa that is in the alert list is cleared or dropped.
The GAL is build up by millicode, when the respective ISC bit is set in the Interruption Alert Mask (IAM) and an interruption of that class is observed.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/interrupt.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 140 insertions(+)arch/s390/kvm/interrupt.c +25
-0@@ -2941,6 +2941,146 @@ int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) return n; } +static int __try_airqs_kick (struct kvm *kvm,u8 ipm) +{ + struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int; + struct kvm_vcpu *vcpu = NULL, *kick_vcpu[MAX_ISC + 1]; + int online_vcpus = atomic_read(&kvm->online_vcpus); + u8 ioint_mask, isc_mask, kick_mask = 0x00; + int vcpu_id, kicked = 0; + + /* Loop over vcpus in WAIT state. */ + for (vcpu_id = find_first_bit(fi->idle_mask, online_vcpus); + /* Until all pending ISCs have a vcpu open for airqs. */ + (~kick_mask & ipm) && vcpu_id < online_vcpus; + vcpu_id = find_next_bit(fi->idle_mask, online_vcpus, vcpu_id)) { + vcpu = kvm_get_vcpu(kvm, vcpu_id); + if (psw_ioint_disabled(vcpu)) + continue; + ioint_mask = (u8)(vcpu->arch.sie_block->gcr[6] >> 24); + for (isc_mask = 0x80; isc_mask; isc_mask >>= 1) { + /* ISC pending in IPM ? */ + if (!(ipm & isc_mask)) + continue; + /* vcpu for this ISC already found ? */ + if (kick_mask & isc_mask) + continue; + /* vcpu open for airq of this ISC ? */Quoted 8 lines
Quoted 8 lines
arch/s390/kvm/interrupt.c int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len) · line 2973
Quoted 5 lines
arch/s390/kvm/interrupt.c +5
-0+ if (!(ioint_mask & isc_mask)) + continue; + /* use this vcpu (for all ISCs in ioint_mask) */ + kick_mask |= ioint_mask; + kick_vcpu[kicked++] = vcpu;Assuming that the vcpu can/will take all ISCs it's currently open for does not seem right. We kind of rely on this assumption here, or?
Quoted 2 lines
why does it not seem right?
Quoted 6 lines
When an interrupt is delivered a psw-swap takes place. The new-psw may fence IO interrupts. Thus for example if we have the vcpu open for all ISCs and 0, 1 and 2 pending, we may end up only delivering 0, if the psw-swap corresponding to delivering 0 closes the vcpu for IO interrupts. After guest has control, we don't have control over the rest of the story.
Quoted 4 lines
OK I think I understand your concern, waking up a single waiting vCPU per ISC is not enough.
We must wake all vCPU in wait state having at least one matching ISC bit.
That is not what I was trying to say, and IMHO generally it also ain't true that we must. But I may be missing something.
Regards,
Halil
PATCH v5 14/15 KVM: s390: add and wire function gib_alert_irq_handler()
In reply to Michael MuellerThe patch implements a handler for GIB alert interruptions on the host. Its task is to alert guests that interrupts are pending for them.
A GIB alert interrupt statistic counter is added as well:
$ cat /proc/interrupts
CPU0 CPU1
...
GAL: 23 37 [I/O] GIB Alert
...--- arch/s390/include/asm/irq.h | 1 + arch/s390/include/asm/isc.h | 1 + arch/s390/kernel/irq.c | 1 + arch/s390/kvm/interrupt.c | 36 ++++++++++++++++++++++++++++++++++-- 4 files changed, 37 insertions(+), 2 deletions(-)
arch/s390/include/asm/irq.h +1 -0
@@ -62,6 +62,7 @@ enum interruption_class {IRQIO_MSI ,IRQIO_VIR ,IRQIO_VAI , +IRQIO_GAL ,NMI_NMI ,CPU_RST ,NR_ARCH_IRQS
arch/s390/include/asm/isc.h +1 -0
@@ -21,6 +21,7 @@/* Adapter interrupts. */ #define QDIO_AIRQ_ISC IO_SCH_ISC/* I/O subchannel in qdio mode */ #define PCI_ISC 2/* PCI I/O subchannels */ +#define GAL_ISC 5/* GIB alert */ #define AP_ISC 6/* adjunct processor (crypto) devices */ /* Functions for registration of I/O interruption subclasses */
arch/s390/kernel/irq.c +1 -0
@@ -88,6 +88,7 @@ static const struct irq_class irqclass_sub_desc[] = { {.irq = IRQIO_MSI, .name = "MSI", .desc = "[I/O] MSI Interrupt" }, {.irq = IRQIO_VIR, .name = "VIR", .desc = "[I/O] Virtual I/O Devices"}, {.irq = IRQIO_VAI, .name = "VAI", .desc = "[I/O] Virtual I/O Devices AI"}, + {.irq = IRQIO_GAL, .name = "GAL", .desc = "[I/O] GIB Alert"}, {.irq = NMI_NMI, .name = "NMI", .desc = "[NMI] Machine Check"}, {.irq = CPU_RST, .name = "RST", .desc = "[CPU] CPU Restart"}, };
arch/s390/kvm/interrupt.c +34 -2
@@ -26,6 +26,7 @@#include <asm/gmap.h> #include <asm/switch_to.h> #include <asm/nmi.h> +#include <asm/airq.h> #include "kvm-s390.h" #include "gaccess.h" #include "trace-s390.h"
@@ -3043,7 +3044,7 @@ static void __floating_airqs_kick(struct kvm *kvm)#define NONE_GISA_ADDR 0x00000001UL#define GISA_ADDR_MASK 0xfffff000UL -static void __maybe_unused process_gib_alert_list (void ) +static void process_gib_alert_list (void ) {u32 final, next_alert, origin =0UL ;struct kvm_s390_gisa *gisa;
@@ -3091,7 +3092,10 @@ void kvm_s390_gisa_clear(struct kvm *kvm) {if (!kvm->arch.gisa)return ; +if (set_iam (kvm->arch.gisa,0 ) == -EBUSY ) +process_gib_alert_list ();nullify_gisa (kvm->arch.gisa); +set_iam (kvm->arch.gisa, kvm->arch.iam);VM_EVENT (kvm,3 ,"gisa 0x%pK cleared" , kvm->arch.gisa); }
@@ -3111,6 +3115,8 @@ void kvm_s390_gisa_destroy(struct kvm *kvm) {if (!kvm->arch.gisa)return ; +if (set_iam (kvm->arch.gisa,0 ) == -EBUSY ) +process_gib_alert_list (); kvm->arch.gisa =NULL ; }
@@ -3159,11 +3165,23 @@ int kvm_s390_gisc_unregister(struct kvm *kvm, u32 gisc) }EXPORT_SYMBOL_GPL (kvm_s390_gisc_unregister); +static void gib_alert_irq_handler (struct airq_struct *airq) +{ +inc_irq_stat (IRQIO_GAL ); +process_gib_alert_list (); +} + +static struct airq_struct gib_alert_irq = { + .handler = gib_alert_irq_handler, + .lsi_ptr = &gib_alert_irq.lsi_mask, +}; +void kvm_s390_gib_destroy (void ) {if (!gib)return ;chsc_sgib (0 ); +unregister_adapter_interrupt (&gib_alert_irq);free_page ((unsigned long )gib); gib =NULL ; }
@@ -3183,16 +3201,30 @@ int kvm_s390_gib_init(u8 nisc) goto out; } + gib_alert_irq.isc = nisc; +if (register_adapter_interrupt (&gib_alert_irq)) { +pr_err ("Registering the GIB alert interruption handler failed\n" ); + rc = -EIO ; + goto out_free; + } + gib->nisc = nisc;if (chsc_sgib ((u32 )(u64 )gib)) {pr_err ("Associating the GIB with the AIV facility failed\n" );free_page ((unsigned long )gib); gib =NULL ; rc = -EIO ; - goto out; + goto out_unreg; }KVM_EVENT (3 ,"gib 0x%pK (nisc=%d) initialized" , gib, gib->nisc); + goto out; + +out_unreg: +unregister_adapter_interrupt (&gib_alert_irq); +out_free: +free_page ((unsigned long )gib); + gib =NULL ; out:return rc; }
Signature · 1 line
2.13.4
Re: [PATCH v5 14/15] KVM: s390: add and wire function gib_alert_irq_handler()
Quoted 78 lines · Michael Mueller
The patch implements a handler for GIB alert interruptions on the host. Its task is to alert guests that interrupts are pending for them.
A GIB alert interrupt statistic counter is added as well:
$ cat /proc/interrupts CPU0 CPU1 ... GAL: 23 37 [I/O] GIB Alert ...Signed-off-by Michael Mueller <[email protected]>--- arch/s390/include/asm/irq.h | 1 + arch/s390/include/asm/isc.h | 1 + arch/s390/kernel/irq.c | 1 + arch/s390/kvm/interrupt.c | 36 ++++++++++++++++++++++++++++++++++-- 4 files changed, 37 insertions(+), 2 deletions(-)arch/s390/include/asm/irq.h +1
-0@@ -62,6 +62,7 @@ enum interruption_class {IRQIO_MSI ,IRQIO_VIR ,IRQIO_VAI , +IRQIO_GAL ,NMI_NMI ,CPU_RST ,NR_ARCH_IRQS arch/s390/include/asm/isc.h +1
-0@@ -21,6 +21,7 @@/* Adapter interrupts. */ #define QDIO_AIRQ_ISC IO_SCH_ISC/* I/O subchannel in qdio mode */ #define PCI_ISC 2/* PCI I/O subchannels */ +#define GAL_ISC 5/* GIB alert */ #define AP_ISC 6/* adjunct processor (crypto) devices */ /* Functions for registration of I/O interruption subclasses */ arch/s390/kernel/irq.c +1
-0@@ -88,6 +88,7 @@ static const struct irq_class irqclass_sub_desc[] = { {.irq = IRQIO_MSI, .name = "MSI", .desc = "[I/O] MSI Interrupt" }, {.irq = IRQIO_VIR, .name = "VIR", .desc = "[I/O] Virtual I/O Devices"}, {.irq = IRQIO_VAI, .name = "VAI", .desc = "[I/O] Virtual I/O Devices AI"}, + {.irq = IRQIO_GAL, .name = "GAL", .desc = "[I/O] GIB Alert"}, {.irq = NMI_NMI, .name = "NMI", .desc = "[NMI] Machine Check"}, {.irq = CPU_RST, .name = "RST", .desc = "[CPU] CPU Restart"}, };arch/s390/kvm/interrupt.c +2
-1@@ -26,6 +26,7 @@#include <asm/gmap.h> #include <asm/switch_to.h> #include <asm/nmi.h> +#include <asm/airq.h> #include "kvm-s390.h" #include "gaccess.h" #include "trace-s390.h" @@ -3043,7 +3044,7 @@ static void __floating_airqs_kick(struct kvm *kvm)#define NONE_GISA_ADDR 0x00000001UL#define GISA_ADDR_MASK 0xfffff000UL -static void __maybe_unused process_gib_alert_list (void ) +static void process_gib_alert_list (void ) {u32 final, next_alert, origin =0UL ;struct kvm_s390_gisa *gisa;
arch/s390/kvm/interrupt.c void kvm_s390_gisa_clear(struct kvm *kvm) · line 3096
Quoted 5 lines
arch/s390/kvm/interrupt.c +2
-0@@ -3091,7 +3092,10 @@ void kvm_s390_gisa_clear(struct kvm *kvm) {if (!kvm->arch.gisa)return ; +if (set_iam (kvm->arch.gisa,0 ) == -EBUSY ) +process_gib_alert_list ();
We call process_gib_alert_list() from different contexts shouldn't we protect the calls?
arch/s390/kvm/interrupt.c void kvm_s390_gisa_destroy(struct kvm *kvm) · line 3119
Quoted 11 lines
arch/s390/kvm/interrupt.c +3
-0nullify_gisa (kvm->arch.gisa); +set_iam (kvm->arch.gisa, kvm->arch.iam);VM_EVENT (kvm,3 ,"gisa 0x%pK cleared" , kvm->arch.gisa); }@@ -3111,6 +3115,8 @@ void kvm_s390_gisa_destroy(struct kvm *kvm) {if (!kvm->arch.gisa)return ; +if (set_iam (kvm->arch.gisa,0 ) == -EBUSY ) +process_gib_alert_list ();
idem.
arch/s390/kvm/interrupt.c int kvm_s390_gisc_unregister(struct kvm *kvm, u32 gisc) · line 3171
Quoted 11 lines
arch/s390/kvm/interrupt.c +4
-0kvm->arch.gisa =NULL ; }@@ -3159,11 +3165,23 @@ int kvm_s390_gisc_unregister(struct kvm *kvm, u32 gisc) }EXPORT_SYMBOL_GPL (kvm_s390_gisc_unregister); +static void gib_alert_irq_handler (struct airq_struct *airq) +{ +inc_irq_stat (IRQIO_GAL ); +process_gib_alert_list ();
idem.
Quoted 49 lines
arch/s390/kvm/interrupt.c +23
-1+} + +static struct airq_struct gib_alert_irq = { + .handler = gib_alert_irq_handler, + .lsi_ptr = &gib_alert_irq.lsi_mask, +}; +void kvm_s390_gib_destroy (void ) {if (!gib)return ;chsc_sgib (0 ); +unregister_adapter_interrupt (&gib_alert_irq);free_page ((unsigned long )gib); gib =NULL ; }@@ -3183,16 +3201,30 @@ int kvm_s390_gib_init(u8 nisc) goto out; } + gib_alert_irq.isc = nisc; +if (register_adapter_interrupt (&gib_alert_irq)) { +pr_err ("Registering the GIB alert interruption handler failed\n" ); + rc = -EIO ; + goto out_free; + } + gib->nisc = nisc;if (chsc_sgib ((u32 )(u64 )gib)) {pr_err ("Associating the GIB with the AIV facility failed\n" );free_page ((unsigned long )gib); gib =NULL ; rc = -EIO ; - goto out; + goto out_unreg; }KVM_EVENT (3 ,"gib 0x%pK (nisc=%d) initialized" , gib, gib->nisc); + goto out; + +out_unreg: +unregister_adapter_interrupt (&gib_alert_irq); +out_free: +free_page ((unsigned long )gib); + gib =NULL ; out:return rc; }
Signature · 2 lines
Pierre Morel Linux/KVM/QEMU in Böblingen - Germany
Re: [PATCH v5 14/15] KVM: s390: add and wire function gib_alert_irq_handler()
Quoted 76 lines · Pierre Morel
Quoted 75 lines · Michael Mueller
The patch implements a handler for GIB alert interruptions on the host. Its task is to alert guests that interrupts are pending for them.
A GIB alert interrupt statistic counter is added as well:
$ cat /proc/interrupts CPU0 CPU1 ... GAL: 23 37 [I/O] GIB Alert ...Signed-off-by Michael Mueller <[email protected]>---
arch/s390/include/asm/irq.h| 1 +arch/s390/include/asm/isc.h| 1 +
arch/s390/kernel/irq.c | 1 + arch/s390/kvm/interrupt.c | 36 ++++++++++++++++++++++++++++++++++--4 files changed, 37 insertions(+), 2 deletions(-)arch/s390/include/asm/irq.h +1
-0@@ -62,6 +62,7 @@ enum interruption_class { IRQIO_MSI, IRQIO_VIR, IRQIO_VAI, + IRQIO_GAL, NMI_NMI, CPU_RST,
NR_ARCH_IRQSarch/s390/include/asm/isc.h +1
-0@@ -21,6 +21,7 @@ /* Adapter interrupts. */ #define QDIO_AIRQ_ISC IO_SCH_ISC /* I/O subchannel in qdio mode */ #define PCI_ISC 2 /* PCI I/O subchannels */ +#define GAL_ISC 5/* GIB alert */ #define AP_ISC 6 /* adjunct processor (crypto) devices */ /* Functions for registration of I/O interruption subclasses */arch/s390/kernel/irq.c +1
-0@@ -88,6 +88,7 @@ static const struct irq_class irqclass_sub_desc[] = { {.irq = IRQIO_MSI, .name = "MSI", .desc = "[I/O] MSI Interrupt" }, {.irq = IRQIO_VIR, .name = "VIR", .desc = "[I/O] Virtual I/O Devices"}, {.irq = IRQIO_VAI, .name = "VAI", .desc = "[I/O] Virtual I/O Devices AI"}, + {.irq = IRQIO_GAL, .name = "GAL", .desc = "[I/O] GIB Alert"}, {.irq = NMI_NMI, .name = "NMI", .desc = "[NMI] Machine Check"}, {.irq = CPU_RST, .name = "RST", .desc = "[CPU] CPU Restart"}, };arch/s390/kvm/interrupt.c +2
-1@@ -26,6 +26,7 @@ #include <asm/gmap.h> #include <asm/switch_to.h> #include <asm/nmi.h> +#include <asm/airq.h> #include "kvm-s390.h" #include "gaccess.h" #include "trace-s390.h"@@ -3043,7 +3044,7 @@ static void __floating_airqs_kick(struct kvm *kvm) #define NONE_GISA_ADDR 0x00000001UL #define GISA_ADDR_MASK 0xfffff000UL -static void __maybe_unused process_gib_alert_list (void ) +static void process_gib_alert_list (void ) { u32 final, next_alert, origin = 0UL; struct kvm_s390_gisa *gisa;
Quoted 8 lines
arch/s390/kvm/interrupt.c void kvm_s390_gisa_clear(struct kvm *kvm) · line 3096
Quoted 6 lines
arch/s390/kvm/interrupt.c +2
-0@@ -3091,7 +3092,10 @@ void kvm_s390_gisa_clear(struct kvm *kvm) { if (!kvm->arch.gisa) return; + if (set_iam(kvm->arch.gisa, 0) == -EBUSY) + process_gib_alert_list();We call
process_gib_alert_list()from different contexts shouldn't we protect the calls?
That should not be necessary as the xcgh() guarantees that both instances will work on gib alert lists with disjunctive gisas.
Quoted 37 lines
arch/s390/kvm/interrupt.c void kvm_s390_gisa_destroy(struct kvm *kvm) · line 3119
Quoted 12 lines
arch/s390/kvm/interrupt.c +3
-0nullify_gisa(kvm->arch.gisa); + set_iam(kvm->arch.gisa, kvm->arch.iam); VM_EVENT(kvm, 3, "gisa 0x%pK cleared", kvm->arch.gisa); }@@ -3111,6 +3115,8 @@ void kvm_s390_gisa_destroy(struct kvm *kvm) { if (!kvm->arch.gisa) return; + if (set_iam(kvm->arch.gisa, 0) == -EBUSY) + process_gib_alert_list();idem.
arch/s390/kvm/interrupt.c int kvm_s390_gisc_unregister(struct kvm *kvm, · line 3171
Quoted 10 lines
arch/s390/kvm/interrupt.c +4
-0kvm->arch.gisa = NULL; }@@ -3159,11 +3165,23 @@ int kvm_s390_gisc_unregister(struct kvm *kvm, u32 gisc) } EXPORT_SYMBOL_GPL(kvm_s390_gisc_unregister); +static void gib_alert_irq_handler (struct airq_struct *airq) +{ + inc_irq_stat(IRQIO_GAL); + process_gib_alert_list();idem.
Quoted 7 lines
arch/s390/kvm/interrupt.c +7
-0+} + +static struct airq_struct gib_alert_irq = { + .handler = gib_alert_irq_handler, + .lsi_ptr = &gib_alert_irq.lsi_mask, +}; +
Re: [PATCH v5 14/15] KVM: s390: add and wire function gib_alert_irq_handler()
Quoted 8 lines · Michael Mueller
Quoted 5 lines · Pierre Morel
Quoted 4 lines · Michael Mueller
The patch implements a handler for GIB alert interruptions on the host. Its task is to alert guests that interrupts are pending for them.
...snip...
Quoted 12 lines
Quoted 11 lines
arch/s390/kvm/interrupt.c void kvm_s390_gisa_clear(struct kvm *kvm) · line 3096
Quoted 9 lines
{
u32 final,next_alert, origin = 0UL;
structkvm_s390_gisa*gisa;arch/s390/kvm/interrupt.c +2
-0@@ -3091,7 +3092,10 @@ void kvm_s390_gisa_clear(struct kvm *kvm) { if (!kvm->arch.gisa) return; + if (set_iam(kvm->arch.gisa, 0) == -EBUSY) + process_gib_alert_list();We call
process_gib_alert_list()from different contexts shouldn't we protect the calls?
Quoted 2 lines
That should not be necessary as the
xcgh()guarantees that both instances will work on gib alert lists with disjunctive gisas.
Here is how I see the problem:
A CPU get the GAL IRQ and start processing the ALERT list.
On another guest we clear floating interrupt...
we call gisa_clear()
we return from set_iam with -EBUSY, meaning the GISA is in alert list.
-> we call process_gib_alert_list()
-> since the list has been disjunct by the GAL IRQ routine we return
immediately
-> we nullify the GISA while it has not been handled by the IRQ
routine
!! if my assumption is right, we loose all GISA following the GISA we just nullified.
Regards,
Pierre
Signature · 2 lines
Pierre Morel Linux/KVM/QEMU in Böblingen - Germany
PATCH v5 15/15 KVM: s390: start using the GIB
In reply to Michael MuellerBy initializing the GIB, it will be used by the kvm host.
--- arch/s390/kvm/kvm-s390.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
arch/s390/kvm/kvm-s390.c +7 -1
@@ -428,16 +428,22 @@ int kvm_arch_init(void *opaque) goto out_debug_unreg; } + rc =kvm_s390_gib_init (GAL_ISC ); +if (rc) + goto out_debug_unreg; +kvm_s390_cpu_feat_init ();/* Register floating interrupt controller interface. */ rc =kvm_register_device_ops (&kvm_flic_ops,KVM_DEV_TYPE_FLIC );if (rc) {pr_err ("Failed to register FLIC rc=%d\n" , rc); - goto out_debug_unreg; + goto out_gib_destroy; }return 0 ; +out_gib_destroy: +kvm_s390_gib_destroy (); out_debug_unreg:debug_unregister (kvm_s390_dbf);return rc;
Signature · 1 line
2.13.4
Re: [PATCH v5 15/15] KVM: s390: start using the GIB
Quoted 12 lines · Michael Mueller
By initializing the GIB, it will be used by the kvm host.
Signed-off-by Michael Mueller <[email protected]>--- arch/s390/kvm/kvm-s390.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)arch/s390/kvm/kvm-s390.c +0
-0
arch/s390/kvm/kvm-s390.c int kvm_arch_init(void *opaque) · line 434
Quoted 7 lines
arch/s390/kvm/kvm-s390.c +4
-0@@ -428,16 +428,22 @@ int kvm_arch_init(void *opaque) goto out_debug_unreg; } + rc =kvm_s390_gib_init (GAL_ISC ); +if (rc) + goto out_debug_unreg; +
Since it is the last introduced I have a preference to put the gib initialization after any existing initialization. it does not have influence them.
arch/s390/kvm/kvm-s390.c int kvm_arch_init(void *opaque) · line 441
Quoted 8 lines
arch/s390/kvm/kvm-s390.c +1
-1kvm_s390_cpu_feat_init ();/* Register floating interrupt controller interface. */ rc =kvm_register_device_ops (&kvm_flic_ops,KVM_DEV_TYPE_FLIC );if (rc) {pr_err ("Failed to register FLIC rc=%d\n" , rc); - goto out_debug_unreg; + goto out_gib_destroy;
It is not a big problem but would make the patch more smaller by gathering all changes.
arch/s390/kvm/kvm-s390.c int kvm_arch_init(void *opaque) · line 449
Quoted 9 lines
arch/s390/kvm/kvm-s390.c +2
-0}return 0 ; +out_gib_destroy: +kvm_s390_gib_destroy (); out_debug_unreg:debug_unregister (kvm_s390_dbf);return rc;
with these changes.
Signature · 2 lines
Pierre Morel Linux/KVM/QEMU in Böblingen - Germany
Re: [PATCH v5 15/15] KVM: s390: start using the GIB
Quoted 31 lines · Pierre Morel
Quoted 12 lines · Michael Mueller
By initializing the GIB, it will be used by the kvm host.
Signed-off-by Michael Mueller <[email protected]>---
arch/s390/kvm/kvm-s390.c| 8 +++++++-1 file changed, 7 insertions(+), 1 deletion(-)arch/s390/kvm/kvm-s390.c +0
-0arch/s390/kvm/kvm-s390.c int kvm_arch_init(void *opaque) · line 433
Quoted 6 lines
arch/s390/kvm/kvm-s390.c +4
-0@@ -428,16 +428,22 @@ int kvm_arch_init(void *opaque) goto out_debug_unreg; } + rc = kvm_s390_gib_init(GAL_ISC); + if (rc) + goto out_debug_unreg; +Since it is the last introduced I have a preference to put the gib initialization after any existing initialization. it does not have influence them.
arch/s390/kvm/kvm-s390.c int kvm_arch_init(void *opaque) · line 441
Quoted 7 lines
arch/s390/kvm/kvm-s390.c +1
-1kvm_s390_cpu_feat_init(); /* Register floating interrupt controller interface. */ rc = kvm_register_device_ops(&kvm_flic_ops, KVM_DEV_TYPE_FLIC); if (rc) { pr_err("Failed to register FLIC rc=%d\n", rc); - goto out_debug_unreg; + goto out_gib_destroy;
Quoted 1 line
It is not a big problem but would make the patch more smaller by gathering all changes.
I will move that behind the FLIC registraion.
Quoted 16 lines
Quoted 9 lines
arch/s390/kvm/kvm-s390.c +2
-0} return 0; +out_gib_destroy: + kvm_s390_gib_destroy();
out_debug_unreg:
debug_unregister(kvm_s390_dbf);
return rc;with these changes.
Reviewed-by Pierre Morel<[email protected]>