KVM: VMX: Translate interrupt shadow when waiting on NMI window

PATCH In review Jan Kiszka 2 participants 17 messages Feb 16, 2010 – May 3, 2010

Subsystems (3)

x86 KVM KVM x86

lore

Share & tools

lore

Jan Kiszka

author

2010-02-16 09:16 UTC #0 lore raw

PATCH 1/1 KVM: VMX: Translate interrupt shadow when waiting on NMI window

Found while browsing Xen code: While we assume that the STI interrupt shadow also inplies virtual NMI blocking, some processors may have a different opinion (SDM 3: 22.3). To avoid misunderstandings that would cause endless VM entry attempts, translate STI into MOV SS blocking when requesting the NMI window.

Signed-off-by Jan Kiszka <[email protected]>

---

 arch/x86/kvm/vmx.c |   15 +++++++++++++++
 1 files changed, 15 insertions(+), 0 deletions(-)

arch/x86/kvm/vmx.c +15 -0

index 14873b9..474f720 100644
@@ -2614,12 +2614,27 @@ static void enable_irq_window(struct kvm_vcpu *vcpu)
 static void enable_nmi_window(struct kvm_vcpu *vcpu)
 {
 	u32 cpu_based_vm_exec_control;
+	u32 interruptibility;
 
 	if (!cpu_has_virtual_nmis()) {
 		enable_irq_window(vcpu);
 		return;
 	}
 
+	/*
+	 * SDM 3: 22.3 (June 2009)
+	 * "A logical processor may also prevent such a VM exit [NMI-window
+	 * exit] if there is blocking of events by STI."
+	 * So better convert STI blocking into MOV SS to avoid premature VM
+	 * exits that would end up in an endless loop.
+	 */
+	interruptibility = vmcs_read32(GUEST_INTERRUPTIBILITY_INFO);
+	if (interruptibility & GUEST_INTR_STATE_STI) {
+		interruptibility &= ~GUEST_INTR_STATE_STI;
+		interruptibility |= GUEST_INTR_STATE_MOV_SS;
+		vmcs_write32(GUEST_INTERRUPTIBILITY_INFO, interruptibility);
+	}
+
 	cpu_based_vm_exec_control = vmcs_read32(CPU_BASED_VM_EXEC_CONTROL);
 	cpu_based_vm_exec_control |= CPU_BASED_VIRTUAL_NMI_PENDING;
 	vmcs_write32(CPU_BASED_VM_EXEC_CONTROL, cpu_based_vm_exec_control);

Gleb Natapov

2010-02-16 10:01 UTC #1 lore raw

Quoted 6 lines · Jan Kiszka

Found while browsing Xen code: While we assume that the STI interrupt shadow also inplies virtual NMI blocking, some processors may have a different opinion (SDM 3: 22.3). To avoid misunderstandings that would cause endless VM entry attempts, translate STI into MOV SS blocking when requesting the NMI window.

Why not just remove "block by STI" check in vmx_nmi_allowed()? IIRC this is documented that on some CPUs STI does not block NMI.

Quoted 28 lines

Signed-off-by Jan Kiszka <[email protected]>

---

 arch/x86/kvm/vmx.c |   15 +++++++++++++++
 1 files changed, 15 insertions(+), 0 deletions(-)

arch/x86/kvm/vmx.c +8 -0

index 14873b9..474f720 100644
@@ -2614,12 +2614,27 @@ static void enable_irq_window(struct kvm_vcpu *vcpu)
 static void enable_nmi_window(struct kvm_vcpu *vcpu)
 {
 	u32 cpu_based_vm_exec_control;
+	u32 interruptibility;
 
 	if (!cpu_has_virtual_nmis()) {
 		enable_irq_window(vcpu);
 		return;
 	}
 
+	/*
+	 * SDM 3: 22.3 (June 2009)
+	 * "A logical processor may also prevent such a VM exit [NMI-window
+	 * exit] if there is blocking of events by STI."
+	 * So better convert STI blocking into MOV SS to avoid premature VM
+	 * exits that would end up in an endless loop.
+	 */

arch/x86/kvm/vmx.c static void enable_irq_window(struct kvm_vcpu *vcpu) · line 2640

Quoted 10 lines

arch/x86/kvm/vmx.c +7 -0

+	interruptibility = vmcs_read32(GUEST_INTERRUPTIBILITY_INFO);
+	if (interruptibility & GUEST_INTR_STATE_STI) {
+		interruptibility &= ~GUEST_INTR_STATE_STI;
+		interruptibility |= GUEST_INTR_STATE_MOV_SS;
+		vmcs_write32(GUEST_INTERRUPTIBILITY_INFO, interruptibility);
+	}
+
 	cpu_based_vm_exec_control = vmcs_read32(CPU_BASED_VM_EXEC_CONTROL);
 	cpu_based_vm_exec_control |= CPU_BASED_VIRTUAL_NMI_PENDING;
 	vmcs_write32(CPU_BASED_VM_EXEC_CONTROL, cpu_based_vm_exec_control);

--
Gleb.

Jan Kiszka

author

2010-02-16 10:04 UTC #2 lore raw

Quoted 10 lines · Gleb Natapov

Quoted 6 lines · Jan Kiszka
Found while browsing Xen code: While we assume that the STI interrupt shadow also inplies virtual NMI blocking, some processors may have a different opinion (SDM 3: 22.3). To avoid misunderstandings that would cause endless VM entry attempts, translate STI into MOV SS blocking when requesting the NMI window.
Why not just remove "block by STI" check in vmx_nmi_allowed()? IIRC this is documented that on some CPUs STI does not block NMI.

Probably because we will stumble and fall on those CPUs that do care.

Jan

Signature · 2 lines

Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Gleb Natapov

2010-02-16 10:06 UTC #3 lore raw

Quoted 8 lines · Jan Kiszka

Quoted 7 lines · Gleb Natapov
Quoted 6 lines · Jan Kiszka
Found while browsing Xen code: While we assume that the STI interrupt shadow also inplies virtual NMI blocking, some processors may have a different opinion (SDM 3: 22.3). To avoid misunderstandings that would cause endless VM entry attempts, translate STI into MOV SS blocking when requesting the NMI window.

Quoted 6 lines

Quoted 3 lines
Why not just remove "block by STI" check in vmx_nmi_allowed()? IIRC this is documented that on some CPUs STI does not block NMI.
Probably because we will stumble and fall on those CPUs that do care.

But this defines behaviour of cpu _we_ emulate. So on _our_ cpu NMI will not be blocked by STI.

--
Gleb.

Jan Kiszka

author

2010-02-16 10:15 UTC #4 lore raw

Quoted 9 lines · Gleb Natapov

Quoted 8 lines · Jan Kiszka
Quoted 7 lines · Gleb Natapov
Quoted 6 lines · Jan Kiszka
Found while browsing Xen code: While we assume that the STI interrupt shadow also inplies virtual NMI blocking, some processors may have a different opinion (SDM 3: 22.3). To avoid misunderstandings that would cause endless VM entry attempts, translate STI into MOV SS blocking when requesting the NMI window.

Quoted 7 lines

Quoted 5 lines
Quoted 3 lines
Why not just remove "block by STI" check in vmx_nmi_allowed()? IIRC this is documented that on some CPUs STI does not block NMI.
Probably because we will stumble and fall on those CPUs that do care.
But this defines behaviour of cpu _we_ emulate. So on _our_ cpu NMI will not be blocked by STI.

The host CPU decides if it accepts an NMI injections while blocked-by-STI is set, not we. But, of course, we could also translate STI into MOV SS blocking unconditionally. Whatever is preferred.

Jan

Signature · 2 lines

Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Gleb Natapov

2010-02-16 10:17 UTC #5 lore raw

Quoted 13 lines · Jan Kiszka

Quoted 12 lines · Gleb Natapov
Quoted 11 lines · Jan Kiszka
Quoted 10 lines · Gleb Natapov
Quoted 6 lines · Jan Kiszka
Found while browsing Xen code: While we assume that the STI interrupt shadow also inplies virtual NMI blocking, some processors may have a different opinion (SDM 3: 22.3). To avoid misunderstandings that would cause endless VM entry attempts, translate STI into MOV SS blocking when requesting the NMI window.
Why not just remove "block by STI" check in vmx_nmi_allowed()? IIRC this is documented that on some CPUs STI does not block NMI.

Quoted 6 lines

Quoted 4 lines
Quoted 2 lines
Probably because we will stumble and fall on those CPUs that do care.
But this defines behaviour of cpu _we_ emulate. So on _our_ cpu NMI will not be blocked by STI.
The host CPU decides if it accepts an NMI injections while

Are you sure? I haven't found such check during VMENTRY.

Quoted 3 lines

blocked-by-STI is set, not we. But, of course, we could also translate STI into MOV SS blocking unconditionally. Whatever is preferred.

--
Gleb.

Jan Kiszka

author

2010-02-16 10:27 UTC #6 lore raw

Quoted 14 lines · Gleb Natapov

Quoted 13 lines · Jan Kiszka
Quoted 12 lines · Gleb Natapov
Quoted 11 lines · Jan Kiszka
Quoted 10 lines · Gleb Natapov
Quoted 6 lines · Jan Kiszka
Found while browsing Xen code: While we assume that the STI interrupt shadow also inplies virtual NMI blocking, some processors may have a different opinion (SDM 3: 22.3). To avoid misunderstandings that would cause endless VM entry attempts, translate STI into MOV SS blocking when requesting the NMI window.
Why not just remove "block by STI" check in vmx_nmi_allowed()? IIRC this is documented that on some CPUs STI does not block NMI.

Quoted 6 lines

Quoted 5 lines
Quoted 4 lines
Quoted 2 lines
Probably because we will stumble and fall on those CPUs that do care.
But this defines behaviour of cpu _we_ emulate. So on _our_ cpu NMI will not be blocked by STI.
The host CPU decides if it accepts an NMI injections while
Are you sure? I haven't found such check during VMENTRY.

I also only find the explicitly stated exclusion of MOV SS blocking vs. NMI injection. If we can rely on this, removing STI blocking from vmx_nmi_allowed should suffice. Or, better, can we get an official confirmation from Intel?

Jan

Signature · 2 lines

Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Gleb Natapov

2010-02-16 10:32 UTC #7 lore raw

Quoted 19 lines · Jan Kiszka

Quoted 18 lines · Gleb Natapov
Quoted 17 lines · Jan Kiszka
Quoted 16 lines · Gleb Natapov
Quoted 8 lines · Jan Kiszka
Quoted 7 lines · Gleb Natapov
Quoted 6 lines · Jan Kiszka
Found while browsing Xen code: While we assume that the STI interrupt shadow also inplies virtual NMI blocking, some processors may have a different opinion (SDM 3: 22.3). To avoid misunderstandings that would cause endless VM entry attempts, translate STI into MOV SS blocking when requesting the NMI window.
Quoted 5 lines
Quoted 3 lines
Why not just remove "block by STI" check in vmx_nmi_allowed()? IIRC this is documented that on some CPUs STI does not block NMI.
Probably because we will stumble and fall on those CPUs that do care.
But this defines behaviour of cpu _we_ emulate. So on _our_ cpu NMI will not be blocked by STI.

Quoted 8 lines

Quoted 2 lines
Quoted 1 line
The host CPU decides if it accepts an NMI injections while
Are you sure? I haven't found such check during VMENTRY.
I also only find the explicitly stated exclusion of MOV SS blocking vs. NMI injection. If we can rely on this, removing STI blocking from vmx_nmi_allowed should suffice. Or, better, can we get an official confirmation from Intel?

SDM 2b says about STI instruction:
The IF flag and the STI and CLI instructions do not prohibit the generation of exceptions and NMI interrupts. NMI interrupts (and SMIs) may be blocked for one macroinstruction following an STI.

--
Gleb.

Jan Kiszka

author

2010-02-16 10:37 UTC #8 lore raw

Quoted 27 lines · Gleb Natapov

Quoted 17 lines · Jan Kiszka
Quoted 16 lines · Gleb Natapov
Quoted 15 lines · Jan Kiszka
Quoted 14 lines · Gleb Natapov
Quoted 13 lines · Jan Kiszka
Quoted 10 lines · Gleb Natapov
Quoted 6 lines · Jan Kiszka
Found while browsing Xen code: While we assume that the STI interrupt shadow also inplies virtual NMI blocking, some processors may have a different opinion (SDM 3: 22.3). To avoid misunderstandings that would cause endless VM entry attempts, translate STI into MOV SS blocking when requesting the NMI window.
Why not just remove "block by STI" check in vmx_nmi_allowed()? IIRC this is documented that on some CPUs STI does not block NMI.
Probably because we will stumble and fall on those CPUs that do care.
Quoted 9 lines
Quoted 4 lines
Quoted 3 lines
Quoted 2 lines
But this defines behaviour of cpu _we_ emulate. So on _our_ cpu NMI will not be blocked by STI.
The host CPU decides if it accepts an NMI injections while
Are you sure? I haven't found such check during VMENTRY.
I also only find the explicitly stated exclusion of MOV SS blocking vs. NMI injection. If we can rely on this, removing STI blocking from vmx_nmi_allowed should suffice. Or, better, can we get an official confirmation from Intel?

Quoted 4 lines

SDM 2b says about STI instruction:
The IF flag and the STI and CLI instructions do not prohibit the generation of exceptions and NMI interrupts. NMI interrupts (and SMIs) may be blocked for one macroinstruction following an STI.

Yes, it's likely that this is the architectural reason for the delayed NMI window signaling after STI. Still, we are looking for the entry-check logic.

Jan

Signature · 2 lines

Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Gleb Natapov

2010-02-16 10:38 UTC #9 lore raw

Quoted 28 lines · Jan Kiszka

Quoted 27 lines · Gleb Natapov
Quoted 26 lines · Jan Kiszka
Quoted 14 lines · Gleb Natapov
Quoted 13 lines · Jan Kiszka
Quoted 12 lines · Gleb Natapov
Quoted 11 lines · Jan Kiszka
Quoted 10 lines · Gleb Natapov
Quoted 6 lines · Jan Kiszka
Found while browsing Xen code: While we assume that the STI interrupt shadow also inplies virtual NMI blocking, some processors may have a different opinion (SDM 3: 22.3). To avoid misunderstandings that would cause endless VM entry attempts, translate STI into MOV SS blocking when requesting the NMI window.
Why not just remove "block by STI" check in vmx_nmi_allowed()? IIRC this is documented that on some CPUs STI does not block NMI.
Quoted 6 lines
Quoted 5 lines
Quoted 4 lines
Quoted 2 lines
Probably because we will stumble and fall on those CPUs that do care.
But this defines behaviour of cpu _we_ emulate. So on _our_ cpu NMI will not be blocked by STI.
The host CPU decides if it accepts an NMI injections while
Are you sure? I haven't found such check during VMENTRY.
I also only find the explicitly stated exclusion of MOV SS blocking vs. NMI injection. If we can rely on this, removing STI blocking from vmx_nmi_allowed should suffice. Or, better, can we get an official confirmation from Intel?

Quoted 9 lines

Quoted 4 lines
SDM 2b says about STI instruction:
The IF flag and the STI and CLI instructions do not prohibit the generation of exceptions and NMI interrupts. NMI interrupts (and SMIs) may be blocked for one macroinstruction following an STI.
Yes, it's likely that this is the architectural reason for the delayed NMI window signaling after STI. Still, we are looking for the entry-check logic.

Will ask Intel.

--
Gleb.

Jan Kiszka

author

2010-04-21 14:17 UTC #10 lore raw

Quoted 33 lines · Gleb Natapov

Quoted 32 lines · Jan Kiszka
Quoted 31 lines · Gleb Natapov
Quoted 17 lines · Jan Kiszka
Quoted 16 lines · Gleb Natapov
Quoted 15 lines · Jan Kiszka
Quoted 14 lines · Gleb Natapov
Quoted 13 lines · Jan Kiszka
Quoted 10 lines · Gleb Natapov
Quoted 6 lines · Jan Kiszka
Found while browsing Xen code: While we assume that the STI interrupt shadow also inplies virtual NMI blocking, some processors may have a different opinion (SDM 3: 22.3). To avoid misunderstandings that would cause endless VM entry attempts, translate STI into MOV SS blocking when requesting the NMI window.
Why not just remove "block by STI" check in vmx_nmi_allowed()? IIRC this is documented that on some CPUs STI does not block NMI.
Probably because we will stumble and fall on those CPUs that do care.
Quoted 9 lines
Quoted 4 lines
Quoted 3 lines
Quoted 2 lines
But this defines behaviour of cpu _we_ emulate. So on _our_ cpu NMI will not be blocked by STI.
The host CPU decides if it accepts an NMI injections while
Are you sure? I haven't found such check during VMENTRY.
I also only find the explicitly stated exclusion of MOV SS blocking vs. NMI injection. If we can rely on this, removing STI blocking from vmx_nmi_allowed should suffice. Or, better, can we get an official confirmation from Intel?
SDM 2b says about STI instruction:
The IF flag and the STI and CLI instructions do not prohibit the generation of exceptions and NMI interrupts. NMI interrupts (and SMIs) may be blocked for one macroinstruction following an STI.

Quoted 6 lines

Quoted 4 lines
Yes, it's likely that this is the architectural reason for the delayed NMI window signaling after STI. Still, we are looking for the entry-check logic.
Will ask Intel.

Just remembered that there was some open topic... Did your ask? Any answer?

Jan

Signature · 2 lines

Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Gleb Natapov

2010-04-21 14:30 UTC #11 lore raw

Quoted 34 lines · Jan Kiszka

Quoted 33 lines · Gleb Natapov
Quoted 32 lines · Jan Kiszka
Quoted 31 lines · Gleb Natapov
Quoted 17 lines · Jan Kiszka
Quoted 16 lines · Gleb Natapov
Quoted 15 lines · Jan Kiszka
Quoted 14 lines · Gleb Natapov
Quoted 13 lines · Jan Kiszka
Quoted 10 lines · Gleb Natapov
Quoted 6 lines · Jan Kiszka
Found while browsing Xen code: While we assume that the STI interrupt shadow also inplies virtual NMI blocking, some processors may have a different opinion (SDM 3: 22.3). To avoid misunderstandings that would cause endless VM entry attempts, translate STI into MOV SS blocking when requesting the NMI window.
Why not just remove "block by STI" check in vmx_nmi_allowed()? IIRC this is documented that on some CPUs STI does not block NMI.
Probably because we will stumble and fall on those CPUs that do care.
Quoted 9 lines
Quoted 4 lines
Quoted 3 lines
Quoted 2 lines
But this defines behaviour of cpu _we_ emulate. So on _our_ cpu NMI will not be blocked by STI.
The host CPU decides if it accepts an NMI injections while
Are you sure? I haven't found such check during VMENTRY.
I also only find the explicitly stated exclusion of MOV SS blocking vs. NMI injection. If we can rely on this, removing STI blocking from vmx_nmi_allowed should suffice. Or, better, can we get an official confirmation from Intel?
SDM 2b says about STI instruction:
The IF flag and the STI and CLI instructions do not prohibit the generation of exceptions and NMI interrupts. NMI interrupts (and SMIs) may be blocked for one macroinstruction following an STI.

Quoted 9 lines

Quoted 6 lines
Quoted 4 lines
Yes, it's likely that this is the architectural reason for the delayed NMI window signaling after STI. Still, we are looking for the entry-check logic.
Will ask Intel.
Just remembered that there was some open topic... Did your ask? Any answer?

I did and got answer last week :) The answer is that NMI is blocked only if GUEST_INTR_STATE_NMI flag is set. MOV SS and STI shouldn't block NMI, so vmx_nmi_allowed() should check only GUEST_INTR_STATE_NMI flag.

--
Gleb.

Jan Kiszka

author

2010-04-21 14:41 UTC #12 lore raw

Quoted 39 lines · Gleb Natapov

Quoted 38 lines · Jan Kiszka
Quoted 37 lines · Gleb Natapov
Quoted 36 lines · Jan Kiszka
Quoted 27 lines · Gleb Natapov
Quoted 17 lines · Jan Kiszka
Quoted 16 lines · Gleb Natapov
Quoted 15 lines · Jan Kiszka
Quoted 14 lines · Gleb Natapov
Quoted 13 lines · Jan Kiszka
Quoted 10 lines · Gleb Natapov
Quoted 6 lines · Jan Kiszka
Found while browsing Xen code: While we assume that the STI interrupt shadow also inplies virtual NMI blocking, some processors may have a different opinion (SDM 3: 22.3). To avoid misunderstandings that would cause endless VM entry attempts, translate STI into MOV SS blocking when requesting the NMI window.
Why not just remove "block by STI" check in vmx_nmi_allowed()? IIRC this is documented that on some CPUs STI does not block NMI.
Probably because we will stumble and fall on those CPUs that do care.
Quoted 9 lines
Quoted 4 lines
Quoted 3 lines
Quoted 2 lines
But this defines behaviour of cpu _we_ emulate. So on _our_ cpu NMI will not be blocked by STI.
The host CPU decides if it accepts an NMI injections while
Are you sure? I haven't found such check during VMENTRY.
I also only find the explicitly stated exclusion of MOV SS blocking vs. NMI injection. If we can rely on this, removing STI blocking from vmx_nmi_allowed should suffice. Or, better, can we get an official confirmation from Intel?
Quoted 4 lines
SDM 2b says about STI instruction:
The IF flag and the STI and CLI instructions do not prohibit the generation of exceptions and NMI interrupts. NMI interrupts (and SMIs) may be blocked for one macroinstruction following an STI.
Yes, it's likely that this is the architectural reason for the delayed NMI window signaling after STI. Still, we are looking for the entry-check logic.

Quoted 7 lines

Quoted 4 lines
Quoted 2 lines
Will ask Intel.
Just remembered that there was some open topic... Did your ask? Any answer?
I did and got answer last week :) The answer is that NMI is blocked only if GUEST_INTR_STATE_NMI flag is set. MOV SS and STI shouldn't block NMI, so vmx_nmi_allowed() should check only GUEST_INTR_STATE_NMI flag.

Cool, that's now increasing my level of confusion again: :(

Thought we only wanted to confirm that it's still safe to inject NMIs when blocked-by-STI is set. Now we hear that it's also safe when MOV SS is active? That would directly contradict the SDM (at least the version I have at hand: June 2009). Or did I misunderstand the answer?

Jan

Signature · 2 lines

Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Gleb Natapov

2010-04-21 14:44 UTC #13 lore raw

Quoted 48 lines · Jan Kiszka

Quoted 39 lines · Gleb Natapov
Quoted 38 lines · Jan Kiszka
Quoted 37 lines · Gleb Natapov
Quoted 36 lines · Jan Kiszka
Quoted 27 lines · Gleb Natapov
Quoted 17 lines · Jan Kiszka
Quoted 16 lines · Gleb Natapov
Quoted 15 lines · Jan Kiszka
Quoted 14 lines · Gleb Natapov
Quoted 13 lines · Jan Kiszka
Quoted 10 lines · Gleb Natapov
Quoted 6 lines · Jan Kiszka
Found while browsing Xen code: While we assume that the STI interrupt
shadow also inplies virtual NMI blocking, some processors may have a
different opinion (SDM 3: 22.3). To avoid misunderstandings that would
cause endless VM entry attempts, translate STI into MOV SS blocking when
requesting the NMI window.
Why not just remove "block by STI" check in vmx_nmi_allowed()? IIRC this is documented that on some CPUs STI does not block NMI.
Probably because we will stumble and fall on those CPUs that do care.
Quoted 9 lines
Quoted 4 lines
Quoted 3 lines
Quoted 2 lines
But this defines behaviour of cpu _we_ emulate. So on _our_ cpu NMI will not be blocked by STI.
The host CPU decides if it accepts an NMI injections while
Are you sure? I haven't found such check during VMENTRY.
I also only find the explicitly stated exclusion of MOV SS blocking vs. NMI injection. If we can rely on this, removing STI blocking from vmx_nmi_allowed should suffice. Or, better, can we get an official confirmation from Intel?
Quoted 4 lines
SDM 2b says about STI instruction:
The IF flag and the STI and CLI instructions do not prohibit the generation of exceptions and NMI interrupts. NMI interrupts (and SMIs) may be blocked for one macroinstruction following an STI.
Yes, it's likely that this is the architectural reason for the delayed NMI window signaling after STI. Still, we are looking for the entry-check logic.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Will ask Intel.
Just remembered that there was some open topic... Did your ask? Any answer?
I did and got answer last week :) The answer is that NMI is blocked only if GUEST_INTR_STATE_NMI flag is set. MOV SS and STI shouldn't block NMI, so vmx_nmi_allowed() should check only GUEST_INTR_STATE_NMI flag.

Quoted 7 lines

Cool, that's now increasing my level of confusion again: :(
Thought we only wanted to confirm that it's still safe to inject NMIs when blocked-by-STI is set. Now we hear that it's also safe when MOV SS is active? That would directly contradict the SDM (at least the version I have at hand: June 2009). Or did I misunderstand the answer?

No you don't. I was told that software should be prepared to handle NMI after MOV SS. What part of SDM does this contradict? I found nothing in latest SDM.

--
Gleb.

Jan Kiszka

author

2010-04-21 15:14 UTC #14 lore raw

Quoted 50 lines · Gleb Natapov

Quoted 49 lines · Jan Kiszka
Quoted 39 lines · Gleb Natapov
Quoted 38 lines · Jan Kiszka
Quoted 37 lines · Gleb Natapov
Quoted 36 lines · Jan Kiszka
Quoted 27 lines · Gleb Natapov
Quoted 17 lines · Jan Kiszka
Quoted 16 lines · Gleb Natapov
Quoted 15 lines · Jan Kiszka
Quoted 14 lines · Gleb Natapov
Quoted 13 lines · Jan Kiszka
Quoted 10 lines · Gleb Natapov
On Tue, Feb 16, 2010 at 10:16:12AM +0100, Jan Kiszka wrote:
Found while browsing Xen code: While we assume that the STI interrupt
shadow also inplies virtual NMI blocking, some processors may have a
different opinion (SDM 3: 22.3). To avoid misunderstandings that would
cause endless VM entry attempts, translate STI into MOV SS blocking when
requesting the NMI window.
Why not just remove "block by STI" check in vmx_nmi_allowed()? IIRC this
is documented that on some CPUs STI does not block NMI.
Probably because we will stumble and fall on those CPUs that do care.
Quoted 9 lines
Quoted 4 lines
Quoted 3 lines
Quoted 2 lines
But this defines behaviour of cpu _we_ emulate. So on _our_ cpu NMI will not be blocked by STI.
The host CPU decides if it accepts an NMI injections while
Are you sure? I haven't found such check during VMENTRY.
I also only find the explicitly stated exclusion of MOV SS blocking vs. NMI injection. If we can rely on this, removing STI blocking from vmx_nmi_allowed should suffice. Or, better, can we get an official confirmation from Intel?
Quoted 4 lines
SDM 2b says about STI instruction:
The IF flag and the STI and CLI instructions do not prohibit the generation of exceptions and NMI interrupts. NMI interrupts (and SMIs) may be blocked for one macroinstruction following an STI.
Yes, it's likely that this is the architectural reason for the delayed NMI window signaling after STI. Still, we are looking for the entry-check logic.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Will ask Intel.
Just remembered that there was some open topic... Did your ask? Any answer?
I did and got answer last week :) The answer is that NMI is blocked only if GUEST_INTR_STATE_NMI flag is set. MOV SS and STI shouldn't block NMI, so vmx_nmi_allowed() should check only GUEST_INTR_STATE_NMI flag.
Cool, that's now increasing my level of confusion again: :(

Quoted 8 lines

Quoted 5 lines
Thought we only wanted to confirm that it's still safe to inject NMIs when blocked-by-STI is set. Now we hear that it's also safe when MOV SS is active? That would directly contradict the SDM (at least the version I have at hand: June 2009). Or did I misunderstand the answer?
No you don't. I was told that software should be prepared to handle NMI after MOV SS. What part of SDM does this contradict? I found nothing in latest SDM.

[ updated to March 2010 version ]

To sum up the scenario again, I think it started with

• If the “NMI-window exiting” VM-execution control is 1, a VM exit occurs before execution of any instruction if there is no virtual-NMI blocking and there is no blocking of events by MOV SS (see Table 21-3). (A logical processor may also prevent such a VM exit if there is blocking of events by STI.) Such a VM exit occurs immediately after VM entry if the above conditions are true (see Section 23.6.6).

We included STI into the NMI shadow, but we /may/ get early exits on some processors according to the statement above. According to your latest info, we can also get that when the MOV SS shadow is on!? But simply allowing NMI injection under MOV SS is not possible:

23.3 CHECKING AND LOADING GUEST STATE
23.3.1.5 Checks on Guest Non-Register State

• Interruptibility state.
...
— Bit 1 (blocking by MOV-SS) must be 0 if the valid bit (bit 31) in the VM-entry
interruption-information field is 1 and the interruption type (bits 10:8) in that field has value 2, indicating non-maskable interrupt (NMI).

And doing this for STI sounds risky too:

— A processor may require bit 0 (blocking by STI) to be 0 if the valid bit (bit 31)
in the VM-entry interruption-information field is 1 and the interruption type (bits 10:8) in that field has value 2, indicating NMI. Other processors may not make this requirement.

Should we start stepping over the shadow like we do for svm?

[ There should be a law that requires hardware builders to write software according to their own manuals... ]

Jan

Signature · 2 lines

Siemens AG, Corporate Technology, CT T DE IT 1
Corporate Competence Center Embedded Linux

Gleb Natapov

2010-04-21 15:30 UTC #15 lore raw

Quoted 89 lines · Jan Kiszka

Quoted 50 lines · Gleb Natapov
Quoted 49 lines · Jan Kiszka
Quoted 39 lines · Gleb Natapov
Quoted 38 lines · Jan Kiszka
Quoted 37 lines · Gleb Natapov
Quoted 36 lines · Jan Kiszka
Quoted 27 lines · Gleb Natapov
Quoted 17 lines · Jan Kiszka
Quoted 16 lines · Gleb Natapov
Quoted 15 lines · Jan Kiszka
Quoted 14 lines · Gleb Natapov
Quoted 13 lines · Jan Kiszka
Gleb Natapov wrote:
On Tue, Feb 16, 2010 at 10:16:12AM +0100, Jan Kiszka wrote:
Found while browsing Xen code: While we assume that the STI interrupt
shadow also inplies virtual NMI blocking, some processors may have a
different opinion (SDM 3: 22.3). To avoid misunderstandings that would
cause endless VM entry attempts, translate STI into MOV SS blocking when
requesting the NMI window.
Why not just remove "block by STI" check in vmx_nmi_allowed()? IIRC this
is documented that on some CPUs STI does not block NMI.
Probably because we will stumble and fall on those CPUs that do care.
Quoted 9 lines
Quoted 4 lines
Quoted 3 lines
Quoted 2 lines
But this defines behaviour of cpu _we_ emulate. So on _our_ cpu NMI will not be blocked by STI.
The host CPU decides if it accepts an NMI injections while
Are you sure? I haven't found such check during VMENTRY.
I also only find the explicitly stated exclusion of MOV SS blocking vs. NMI injection. If we can rely on this, removing STI blocking from vmx_nmi_allowed should suffice. Or, better, can we get an official confirmation from Intel?
Quoted 4 lines
SDM 2b says about STI instruction:
The IF flag and the STI and CLI instructions do not prohibit the generation of exceptions and NMI interrupts. NMI interrupts (and SMIs) may be blocked for one macroinstruction following an STI.
Yes, it's likely that this is the architectural reason for the delayed NMI window signaling after STI. Still, we are looking for the entry-check logic.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Will ask Intel.
Just remembered that there was some open topic... Did your ask? Any answer?
I did and got answer last week :) The answer is that NMI is blocked only if GUEST_INTR_STATE_NMI flag is set. MOV SS and STI shouldn't block NMI, so vmx_nmi_allowed() should check only GUEST_INTR_STATE_NMI flag.
Cool, that's now increasing my level of confusion again: :(
Quoted 8 lines
Quoted 5 lines
Thought we only wanted to confirm that it's still safe to inject NMIs when blocked-by-STI is set. Now we hear that it's also safe when MOV SS is active? That would directly contradict the SDM (at least the version I have at hand: June 2009). Or did I misunderstand the answer?
No you don't. I was told that software should be prepared to handle NMI after MOV SS. What part of SDM does this contradict? I found nothing in latest SDM.
[ updated to March 2010 version ]
To sum up the scenario again, I think it started with
• If the “NMI-window exiting” VM-execution control is 1, a VM exit occurs before execution of any instruction if there is no virtual-NMI blocking and there is no blocking of events by MOV SS (see Table 21-3). (A logical processor may also prevent such a VM exit if there is blocking of events by STI.) Such a VM exit occurs immediately after VM entry if the above conditions are true (see Section 23.6.6).
We included STI into the NMI shadow, but we /may/ get early exits on some processors according to the statement above. According to your latest info, we can also get that when the MOV SS shadow is on!? But simply allowing NMI injection under MOV SS is not possible:
23.3 CHECKING AND LOADING GUEST STATE
23.3.1.5 Checks on Guest Non-Register State
• Interruptibility state.
...
— Bit 1 (blocking by MOV-SS) must be 0 if the valid bit (bit 31) in the VM-entry
interruption-information field is 1 and the interruption type (bits 10:8) in that field has value 2, indicating non-maskable interrupt (NMI).
And doing this for STI sounds risky too:

Quoted 8 lines

— A processor may require bit 0 (blocking by STI) to be 0 if the valid bit (bit 31)
in the VM-entry interruption-information field is 1 and the interruption type (bits 10:8) in that field has value 2, indicating NMI. Other processors may not make this requirement.
Should we start stepping over the shadow like we do for svm?

If x86 ISA allows NMI to be injected after STI and MOV SS we can clear blocking by STI/MOV SS bits before injecting NMI. But why would Intel add those checks then. Will ask Intel once again. Hope will get response sooner now.

Quoted 3 lines

[ There should be a law that requires hardware builders to write software according to their own manuals... ]

--
Gleb.

Gleb Natapov

2010-05-03 07:32 UTC #16 lore raw

In reply to Jan Kiszka

Quoted 33 lines · Jan Kiszka

Quoted 3 lines
No you don't. I was told that software should be prepared to handle NMI after MOV SS. What part of SDM does this contradict? I found nothing in latest SDM.
[ updated to March 2010 version ]
To sum up the scenario again, I think it started with
• If the “NMI-window exiting” VM-execution control is 1, a VM exit occurs before execution of any instruction if there is no virtual-NMI blocking and there is no blocking of events by MOV SS (see Table 21-3). (A logical processor may also prevent such a VM exit if there is blocking of events by STI.) Such a VM exit occurs immediately after VM entry if the above conditions are true (see Section 23.6.6).
We included STI into the NMI shadow, but we /may/ get early exits on some processors according to the statement above. According to your latest info, we can also get that when the MOV SS shadow is on!? But simply allowing NMI injection under MOV SS is not possible:
23.3 CHECKING AND LOADING GUEST STATE
23.3.1.5 Checks on Guest Non-Register State
• Interruptibility state.
...
— Bit 1 (blocking by MOV-SS) must be 0 if the valid bit (bit 31) in the VM-entry
interruption-information field is 1 and the interruption type (bits 10:8) in that field has value 2, indicating non-maskable interrupt (NMI).
And doing this for STI sounds risky too:

Quoted 8 lines

— A processor may require bit 0 (blocking by STI) to be 0 if the valid bit (bit 31)
in the VM-entry interruption-information field is 1 and the interruption type (bits 10:8) in that field has value 2, indicating NMI. Other processors may not make this requirement.
Should we start stepping over the shadow like we do for svm?

Intel's answer is that text above describes model-specific behaviour in bare metal which can block NMI for one instruction after STI/MOV SS, but since software should not rely on model-specific behaviour we can safely inject NMI after STI/MOV SS (clearing "blocked by STI/MOV SS" bit before injecting).

--
Gleb.