x86/hvm: Multiple corrections to task switch handling
v2 cover x86/hvm: Multiple corrections to task switch handling
These patches want backporting due to the severity of patch 2. They should therefore be considered for 4.13 at this point.
Andrew Cooper (3):
x86/vtx: Fix fault semantics for early task switch failures x86/svm: Always intercept ICEBP
x86/svm: Write the correct %eip into the outgoing task
xen/arch/x86/hvm/hvm.c | 4 +- xen/arch/x86/hvm/svm/emulate.c | 54 ++++++++++++++++++++++++ xen/arch/x86/hvm/svm/svm.c | 77 ++++++++++++++++++++++------------- xen/arch/x86/hvm/svm/vmcb.c | 2 +- xen/arch/x86/hvm/vmx/vmx.c | 4 +- xen/arch/x86/monitor.c | 3 -- xen/include/asm-x86/hvm/hvm.h | 13 +----- xen/include/asm-x86/hvm/svm/emulate.h | 1 + 8 files changed, 109 insertions(+), 49 deletions(-)
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 1/3 x86/vtx: Fix fault semantics for early task switch failures
The VT-x task switch handler adds inst_len to %rip before calling hvm_task_switch(), which is problematic in two ways:
1) Early faults (i.e. ones delivered in the context of the old task) get delivered with trap semantics, and break restartibility.
2) The addition isn't truncated to 32 bits. In the corner case of a task switch instruction crossing the 4G->0 boundary taking an early fault (with trap semantics), a VMEntry failure will occur due to %rip being out of range.
Instead, pass the instruction length into hvm_task_switch() and write it into the outgoing TSS only, leaving %rip in its original location.
For now, pass 0 on the SVM side. This highlights a separate preexisting bug which will be addressed in the following patch.
While adjusting call sites, drop the unnecessary uint16_t cast.
--- xen/arch/x86/hvm/hvm.c | 4 ++-- xen/arch/x86/hvm/svm/svm.c | 2 +- xen/arch/x86/hvm/vmx/vmx.c | 4 ++-- xen/include/asm-x86/hvm/hvm.h | 2 +- 4 files changed, 6 insertions(+), 6 deletions(-)
xen/arch/x86/hvm/hvm.c +2 -2
@@ -2913,7 +2913,7 @@ void hvm_prepare_vm86_tss(struct vcpu *v, uint32_t base, uint32_t limit)void hvm_task_switch (uint16_t tss_sel,enum hvm_task_switch_reason taskswitch_reason, -int32_t errcode) +int32_t errcode,unsigned int insn_len) {struct vcpu *v = current;struct cpu_user_regs *regs =guest_cpu_user_regs ();
@@ -2987,7 +2987,7 @@ void hvm_task_switch(if ( taskswitch_reason == TSW_iret ) eflags &= ~X86_EFLAGS_NT ; - tss.eip = regs->eip; + tss.eip = regs->eip + insn_len; tss.eflags = eflags; tss.eax = regs->eax; tss.ecx = regs->ecx;
xen/arch/x86/hvm/svm/svm.c +1 -1
@@ -2794,7 +2794,7 @@ void svm_vmexit_handler(struct cpu_user_regs *regs)*/ vmcb->eventinj.bytes =0 ; -hvm_task_switch ((uint16_t )vmcb->exitinfo1, reason, errcode); +hvm_task_switch (vmcb->exitinfo1, reason, errcode,0 );break ; }
xen/arch/x86/hvm/vmx/vmx.c +2 -2
@@ -3962,8 +3962,8 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)__vmread (IDT_VECTORING_ERROR_CODE , &ecode); else ecode =-1 ; - regs->rip += inst_len; -hvm_task_switch ((uint16_t )exit_qualification, reasons[source], ecode); + +hvm_task_switch (exit_qualification, reasons[source], ecode, inst_len);break ; }case EXIT_REASON_CPUID :
xen/include/asm-x86/hvm/hvm.h +1 -1
@@ -297,7 +297,7 @@ void hvm_set_rdtsc_exiting(struct domain *d, bool_t enable);enum hvm_task_switch_reason { TSW_jmp, TSW_iret, TSW_call_or_int };void hvm_task_switch (uint16_t tss_sel,enum hvm_task_switch_reason taskswitch_reason, -int32_t errcode); +int32_t errcode,unsigned int insn_len);enum hvm_access_type { hvm_access_insn_fetch,
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 2/3 x86/svm: Always intercept ICEBP
In reply to Andrew CooperICEBP isn't handled well by SVM.
The VMexit state for a #DB-vectored TASK_SWITCH has %rip pointing to the appropriate instruction boundary (fault or trap, as appropriate), except for an ICEBP-induced #DB TASK_SWITCH, where %rip points at the ICEBP instruction rather than after it. As ICEBP isn't distinguished in the vectoring event type, the state is ambiguous.
To add to the confusion, an ICEBP which occurs due to Introspection intercepting the instruction, or from x86_emulate() will have %rip updated as a consequence of partial emulation required to inject an ICEBP event in the first place.
We could in principle spot the non-injected case in the TASK_SWITCH handler, but this still results in complexity if the ICEBP instruction also has an Instruction Breakpoint active on it (which genuinely has fault semantics).
Unconditionally intercept ICEBP. This does have a trap semantics for the intercept, and allows us to move %rip forwards appropriately before the TASK_SWITCH intercept is hit. This makes the behaviour of #DB-vectored switches consistent however the ICEBP #DB came about, and avoids special cases in the TASK_SWITCH intercept.
This in turn allows for the removal of the conditional hvm_set_icebp_interception() logic used by the monitor subsystem, as ICEBP's will now always be submitted for monitoring checks.
---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Razvan Cojocaru <[email protected]>
CC: Tamas K Lengyel <[email protected]>
CC: Alexandru Isaila <[email protected]>
CC: Petre Pircalabu <[email protected]>
CC: Juergen Gross <[email protected]>
v2:
* New
--- xen/arch/x86/hvm/svm/svm.c | 19 ------------------- xen/arch/x86/hvm/svm/vmcb.c | 2 +- xen/arch/x86/monitor.c | 3 --- xen/include/asm-x86/hvm/hvm.h | 11 ----------- 4 files changed, 1 insertion(+), 34 deletions(-)
xen/arch/x86/hvm/svm/svm.c +0 -19
@@ -173,24 +173,6 @@ static void svm_enable_msr_interception(struct domain *d, uint32_t msr)svm_intercept_msr (v, msr,MSR_INTERCEPT_WRITE ); } -static void svm_set_icebp_interception (struct domain *d,bool enable) -{ -const struct vcpu *v; - - for_each_vcpu ( d, v ) - { -struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb; -uint32_t intercepts =vmcb_get_general2_intercepts (vmcb); - -if ( enable ) - intercepts |=GENERAL2_INTERCEPT_ICEBP ; -else - intercepts &= ~GENERAL2_INTERCEPT_ICEBP ; - -vmcb_set_general2_intercepts (vmcb, intercepts); - } -} -static void svm_save_dr (struct vcpu *v) {struct vmcb_struct *vmcb = v->arch.hvm.svm.vmcb;
@@ -2474,7 +2456,6 @@ static struct hvm_function_table __initdata svm_function_table = { .msr_read_intercept = svm_msr_read_intercept, .msr_write_intercept = svm_msr_write_intercept, .enable_msr_interception = svm_enable_msr_interception, - .set_icebp_interception = svm_set_icebp_interception, .set_rdtsc_exiting = svm_set_rdtsc_exiting, .set_descriptor_access_exiting = svm_set_descriptor_access_exiting, .get_insn_bytes = svm_get_insn_bytes,
xen/arch/x86/hvm/svm/vmcb.c +1 -1
@@ -73,7 +73,7 @@ static int construct_vmcb(struct vcpu *v)GENERAL2_INTERCEPT_STGI |GENERAL2_INTERCEPT_CLGI |GENERAL2_INTERCEPT_SKINIT |GENERAL2_INTERCEPT_MWAIT |GENERAL2_INTERCEPT_WBINVD |GENERAL2_INTERCEPT_MONITOR | -GENERAL2_INTERCEPT_XSETBV ; +GENERAL2_INTERCEPT_XSETBV |GENERAL2_INTERCEPT_ICEBP ;/* Intercept all debug-register writes. */ vmcb->_dr_intercepts = ~0u ;
xen/arch/x86/monitor.c +0 -3
@@ -301,9 +301,6 @@ int arch_monitor_domctl_event(struct domain *d, ad->monitor.debug_exception_sync = requested_status ? mop->u.debug_exception.sync :0 ; - -hvm_set_icebp_interception (d, requested_status); -domain_unpause (d);break ; }
xen/include/asm-x86/hvm/hvm.h +0 -11
@@ -206,7 +206,6 @@ struct hvm_function_table { bool_t access_w, bool_t access_x);void (*enable_msr_interception)(struct domain *d,uint32_t msr); -void (*set_icebp_interception)(struct domain *d,bool enable);bool_t (*is_singlestep_supported)(void);/* Alternate p2m */
@@ -615,16 +614,6 @@ static inline bool_t hvm_enable_msr_interception(struct domain *d, uint32_t msr)return 0 ; } -static inline bool hvm_set_icebp_interception (struct domain *d,bool enable) -{ -if ( hvm_funcs.set_icebp_interception ) - { - hvm_funcs.set_icebp_interception (d, enable); -return true; - } -return false; -} -static inline bool_t hvm_is_singlestep_supported (void ) { return (hvm_funcs.is_singlestep_supported &&
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 2/3] x86/svm: Always intercept ICEBP
Quoted 23 lines · Andrew Cooper
ICEBP isn't handled well by SVM.
The VMexit state for a #DB-vectored
TASK_SWITCHhas %rip pointing to the appropriate instruction boundary (fault or trap, as appropriate), except for an ICEBP-induced #DBTASK_SWITCH, where %rip points at the ICEBP instruction rather than after it. As ICEBP isn't distinguished in the vectoring event type, the state is ambiguous.To add to the confusion, an ICEBP which occurs due to Introspection intercepting the instruction, or from
x86_emulate()will have %rip updated as a consequence of partial emulation required to inject an ICEBP event in the first place.We could in principle spot the non-injected case in the
TASK_SWITCHhandler, but this still results in complexity if the ICEBP instruction also has an Instruction Breakpoint active on it (which genuinely has fault semantics).Unconditionally intercept ICEBP. This does have a trap semantics for the intercept, and allows us to move %rip forwards appropriately before the
TASK_SWITCHintercept is hit. This makes the behaviour of #DB-vectored switches consistent however the ICEBP #DB came about, and avoids special cases in theTASK_SWITCHintercept.
Quoted 5 lines
This in turn allows for the removal of the conditional
hvm_set_icebp_interception()logic used by the monitor subsystem, as ICEBP's will now always be submitted for monitoring checks.Signed-off-by Andrew Cooper <[email protected]>
LGTM.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 2/3] x86/svm: Always intercept ICEBP
In reply to Andrew CooperQuoted 36 lines · Andrew Cooper
ICEBP isn't handled well by SVM.
The VMexit state for a #DB-vectored
TASK_SWITCHhas %rip pointing to the
appropriate instruction boundary (fault or trap, as appropriate), except for
an ICEBP-induced #DBTASK_SWITCH, where %rip points at the ICEBP instruction
rather than after it. As ICEBP isn't distinguished in the vectoring event
type, the state is ambiguous.To add to the confusion, an ICEBP which occurs due to Introspection intercepting the instruction, or from
x86_emulate()will have %rip updated as
a consequence of partial emulation required to inject an ICEBP event in the
first place.We could in principle spot the non-injected case in the
TASK_SWITCHhandler,
but this still results in complexity if the ICEBP instruction also has an
Instruction Breakpoint active on it (which genuinely has fault semantics).Unconditionally intercept ICEBP. This does have a trap semantics for the
intercept, and allows us to move %rip forwards appropriately before theTASK_SWITCHintercept is hit. This makes the behaviour of #DB- vectored
switches consistent however the ICEBP #DB came about, and avoids special cases
in theTASK_SWITCHintercept.
Quoted 7 lines
This in turn allows for the removal of the conditional
hvm_set_icebp_interception()logic used by the monitor subsystem, as ICEBP's
will now always be submitted for monitoring checks.Signed-off-by Andrew Cooper <[email protected]>
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 2/3] x86/svm: Always intercept ICEBP
In reply to Andrew CooperQuoted 13 lines · Andrew Cooper
ICEBP isn't handled well by SVM.
The VMexit state for a #DB-vectored
TASK_SWITCHhas %rip pointing to the appropriate instruction boundary (fault or trap, as appropriate), except for an ICEBP-induced #DBTASK_SWITCH, where %rip points at the ICEBP instruction rather than after it. As ICEBP isn't distinguished in the vectoring event type, the state is ambiguous.To add to the confusion, an ICEBP which occurs due to Introspection intercepting the instruction, or from
x86_emulate()will have %rip updated as a consequence of partial emulation required to inject an ICEBP event in the first place.
Quoted 7 lines
We could in principle spot the non-injected case in the
TASK_SWITCHhandler, but this still results in complexity if the ICEBP instruction also has an Instruction Breakpoint active on it (which genuinely has fault semantics).Unconditionally intercept ICEBP. This does have a trap semantics for the intercept, and allows us to move %rip forwards appropriately before the
TASK_SWITCHintercept is hit.
Both because of you mentioning the moving forwards of %rip and with the irc discussion in mind that we had no irc, don't you mean "fault semantics" here? If so
Otherwise I guess I'm still missing something.
Quoted 6 lines
--- xen/arch/x86/hvm/svm/svm.c | 19 ------------------- xen/arch/x86/hvm/svm/vmcb.c | 2 +- xen/arch/x86/monitor.c | 3 --- xen/include/asm-x86/hvm/hvm.h | 11 ----------- 4 files changed, 1 insertion(+), 34 deletions(-)
This, of course, is pretty nice in any event.
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 2/3] x86/svm: Always intercept ICEBP
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Andrew Cooper
ICEBP isn't handled well by SVM.
The VMexit state for a #DB-vectored
TASK_SWITCHhas %rip pointing to the appropriate instruction boundary (fault or trap, as appropriate), except for an ICEBP-induced #DBTASK_SWITCH, where %rip points at the ICEBP instruction rather than after it. As ICEBP isn't distinguished in the vectoring event type, the state is ambiguous.To add to the confusion, an ICEBP which occurs due to Introspection intercepting the instruction, or from
x86_emulate()will have %rip updated as a consequence of partial emulation required to inject an ICEBP event in the first place.We could in principle spot the non-injected case in the
TASK_SWITCHhandler, but this still results in complexity if the ICEBP instruction also has an Instruction Breakpoint active on it (which genuinely has fault semantics).
Quoted 6 lines
Quoted 3 lines
Unconditionally intercept ICEBP. This does have a trap semantics for the intercept, and allows us to move %rip forwards appropriately before the
TASK_SWITCHintercept is hit.Both because of you mentioning the moving forwards of %rip and with the irc discussion in mind that we had no irc, don't you mean "fault semantics" here?
ICEBP really is too broken under SVM to handle architecturally.
The ICEBP intercept has nRIP decode support, because it is an instruction intercept. We emulate the injection (because it is ICEBP), which means we re-enter the guest with %rip moved forward, and #DB (HW_EXCEPTION) pending for injection. This means that...
Quoted 2 lines
If so
Reviewed-by Jan Beulich <[email protected]>
... the ICEBP-#DB-vectored TASK_SWITCH will now find %rip pointing after the ICEBP instruction, rather than at it, making it consistent with every other #DB-vectored TASK_SWITCH.
This does means that an early task-switch fault for ICEBP will reliably be delivered with the wrong (i.e. trap) semantics, but this is less bad than mixed fault/trap semantics depending on whether the source of the ICEBP was introspection/emulation or native execution.
We could restore proper fault behaviour by extending svm_emul_swint_injection() to figure out that a task switch is needed, and invoke hvm_task_switch() directly, but I don't have enough TUITS right now.
Quoted 1 line
Otherwise I guess I'm still missing something.
I hope this clears it up.
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 2/3] x86/svm: Always intercept ICEBP
Quoted 45 lines · Andrew Cooper
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Andrew Cooper
ICEBP isn't handled well by SVM.
The VMexit state for a #DB-vectored
TASK_SWITCHhas %rip pointing to the appropriate instruction boundary (fault or trap, as appropriate), except for an ICEBP-induced #DBTASK_SWITCH, where %rip points at the ICEBP instruction rather than after it. As ICEBP isn't distinguished in the vectoring event type, the state is ambiguous.To add to the confusion, an ICEBP which occurs due to Introspection intercepting the instruction, or from
x86_emulate()will have %rip updated as a consequence of partial emulation required to inject an ICEBP event in the first place.We could in principle spot the non-injected case in the
TASK_SWITCHhandler, but this still results in complexity if the ICEBP instruction also has an Instruction Breakpoint active on it (which genuinely has fault semantics).Quoted 6 lines
Quoted 3 lines
Unconditionally intercept ICEBP. This does have a trap semantics for the intercept, and allows us to move %rip forwards appropriately before the
TASK_SWITCHintercept is hit.Both because of you mentioning the moving forwards of %rip and with the irc discussion in mind that we had no irc, don't you mean "fault semantics" here?
ICEBP really is too broken under SVM to handle architecturally.
The ICEBP intercept has nRIP decode support, because it is an instruction intercept. We emulate the injection (because it is ICEBP), which means we re-enter the guest with %rip moved forward, and #DB (
HW_EXCEPTION) pending for injection. This means that...Quoted 2 lines
If so
Reviewed-by Jan Beulich <[email protected]>... the ICEBP-#DB-vectored
TASK_SWITCHwill now find %rip pointing after the ICEBP instruction, rather than at it, making it consistent with every other #DB-vectoredTASK_SWITCH.This does means that an early task-switch fault for ICEBP will reliably be delivered with the wrong (i.e. trap) semantics, but this is less bad than mixed fault/trap semantics depending on whether the source of the ICEBP was introspection/emulation or native execution.
Quoted 8 lines
We could restore proper fault behaviour by extending
svm_emul_swint_injection()to figure out that a task switch is needed, and invokehvm_task_switch()directly, but I don't have enough TUITS right now.Quoted 1 line
Otherwise I guess I'm still missing something.
I hope this clears it up.
Well, it helps, but you don't really answer the question: Is "trap" in that sentence of the description really correct? I.e. don't you instead mean "fault" there?
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 2/3] x86/svm: Always intercept ICEBP
Quoted 49 lines · Jan Beulich
Quoted 48 lines · Andrew Cooper
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Andrew Cooper
ICEBP isn't handled well by SVM.
The VMexit state for a #DB-vectored
TASK_SWITCHhas %rip pointing to the appropriate instruction boundary (fault or trap, as appropriate), except for an ICEBP-induced #DBTASK_SWITCH, where %rip points at the ICEBP instruction rather than after it. As ICEBP isn't distinguished in the vectoring event type, the state is ambiguous.To add to the confusion, an ICEBP which occurs due to Introspection intercepting the instruction, or from
x86_emulate()will have %rip updated as a consequence of partial emulation required to inject an ICEBP event in the first place.We could in principle spot the non-injected case in the
TASK_SWITCHhandler, but this still results in complexity if the ICEBP instruction also has an Instruction Breakpoint active on it (which genuinely has fault semantics).Quoted 6 lines
Quoted 3 lines
Unconditionally intercept ICEBP. This does have a trap semantics for the intercept, and allows us to move %rip forwards appropriately before the
TASK_SWITCHintercept is hit.Both because of you mentioning the moving forwards of %rip and with the irc discussion in mind that we had no irc, don't you mean "fault semantics" here?
ICEBP really is too broken under SVM to handle architecturally.
The ICEBP intercept has nRIP decode support, because it is an instruction intercept. We emulate the injection (because it is ICEBP), which means we re-enter the guest with %rip moved forward, and #DB (
HW_EXCEPTION) pending for injection. This means that...Quoted 2 lines
If so
Reviewed-by Jan Beulich <[email protected]>... the ICEBP-#DB-vectored
TASK_SWITCHwill now find %rip pointing after the ICEBP instruction, rather than at it, making it consistent with every other #DB-vectoredTASK_SWITCH.This does means that an early task-switch fault for ICEBP will reliably be delivered with the wrong (i.e. trap) semantics, but this is less bad than mixed fault/trap semantics depending on whether the source of the ICEBP was introspection/emulation or native execution.
We could restore proper fault behaviour by extending
svm_emul_swint_injection()to figure out that a task switch is needed, and invokehvm_task_switch()directly, but I don't have enough TUITS right now.
Quoted 5 lines
Quoted 2 lines
Quoted 1 line
Otherwise I guess I'm still missing something.
I hope this clears it up.
Well, it helps, but you don't really answer the question: Is "trap" in that sentence of the description really correct? I.e. don't you instead mean "fault" there?
I've reworded that bit to:
Unconditionally intercept ICEBP. This does have NRIPs support as it is an instruction intercept, which allows us allows us to move %rip forwards appropriately before the TASK_SWITCH intercept is hit. This allows...
Any better?
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 2/3] x86/svm: Always intercept ICEBP
Quoted 56 lines · Andrew Cooper
Quoted 49 lines · Jan Beulich
Quoted 48 lines · Andrew Cooper
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Andrew Cooper
ICEBP isn't handled well by SVM.
The VMexit state for a #DB-vectored
TASK_SWITCHhas %rip pointing to the appropriate instruction boundary (fault or trap, as appropriate), except for an ICEBP-induced #DBTASK_SWITCH, where %rip points at the ICEBP instruction rather than after it. As ICEBP isn't distinguished in the vectoring event type, the state is ambiguous.To add to the confusion, an ICEBP which occurs due to Introspection intercepting the instruction, or from
x86_emulate()will have %rip updated as a consequence of partial emulation required to inject an ICEBP event in the first place.We could in principle spot the non-injected case in the
TASK_SWITCHhandler, but this still results in complexity if the ICEBP instruction also has an Instruction Breakpoint active on it (which genuinely has fault semantics).Quoted 6 lines
Quoted 3 lines
Unconditionally intercept ICEBP. This does have a trap semantics for the intercept, and allows us to move %rip forwards appropriately before the
TASK_SWITCHintercept is hit.Both because of you mentioning the moving forwards of %rip and with the irc discussion in mind that we had no irc, don't you mean "fault semantics" here?
ICEBP really is too broken under SVM to handle architecturally.
The ICEBP intercept has nRIP decode support, because it is an instruction intercept. We emulate the injection (because it is ICEBP), which means we re-enter the guest with %rip moved forward, and #DB (
HW_EXCEPTION) pending for injection. This means that...Quoted 2 lines
If so
Reviewed-by Jan Beulich <[email protected]>... the ICEBP-#DB-vectored
TASK_SWITCHwill now find %rip pointing after the ICEBP instruction, rather than at it, making it consistent with every other #DB-vectoredTASK_SWITCH.This does means that an early task-switch fault for ICEBP will reliably be delivered with the wrong (i.e. trap) semantics, but this is less bad than mixed fault/trap semantics depending on whether the source of the ICEBP was introspection/emulation or native execution.
We could restore proper fault behaviour by extending
svm_emul_swint_injection()to figure out that a task switch is needed, and invokehvm_task_switch()directly, but I don't have enough TUITS right now.Quoted 5 lines
Quoted 2 lines
Quoted 1 line
Otherwise I guess I'm still missing something.
I hope this clears it up.
Well, it helps, but you don't really answer the question: Is "trap" in that sentence of the description really correct? I.e. don't you instead mean "fault" there?
Quoted 7 lines
I've reworded that bit to:
Unconditionally intercept ICEBP. This does have NRIPs support as it is an instruction intercept, which allows us allows us to move %rip forwards appropriately before the
TASK_SWITCHintercept is hit. This allows...Any better?
Ah yes, thanks. (But please drop one of the two "allows us".)
Jan
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 2/3] x86/svm: Always intercept ICEBP
Quoted 56 lines · Jan Beulich
Quoted 55 lines · Andrew Cooper
Quoted 54 lines · Jan Beulich
Quoted 43 lines · Andrew Cooper
Quoted 18 lines · Jan Beulich
Quoted 17 lines · Andrew Cooper
ICEBP isn't handled well by SVM.
The VMexit state for a #DB-vectored
TASK_SWITCHhas %rip pointing to the appropriate instruction boundary (fault or trap, as appropriate), except for an ICEBP-induced #DBTASK_SWITCH, where %rip points at the ICEBP instruction rather than after it. As ICEBP isn't distinguished in the vectoring event type, the state is ambiguous.To add to the confusion, an ICEBP which occurs due to Introspection intercepting the instruction, or from
x86_emulate()will have %rip updated as a consequence of partial emulation required to inject an ICEBP event in the first place.We could in principle spot the non-injected case in the
TASK_SWITCHhandler, but this still results in complexity if the ICEBP instruction also has an Instruction Breakpoint active on it (which genuinely has fault semantics).Quoted 6 lines
Quoted 3 lines
Unconditionally intercept ICEBP. This does have a trap semantics for the intercept, and allows us to move %rip forwards appropriately before the
TASK_SWITCHintercept is hit.Both because of you mentioning the moving forwards of %rip and with the irc discussion in mind that we had no irc, don't you mean "fault semantics" here?
ICEBP really is too broken under SVM to handle architecturally.
The ICEBP intercept has nRIP decode support, because it is an instruction intercept. We emulate the injection (because it is ICEBP), which means we re-enter the guest with %rip moved forward, and #DB (
HW_EXCEPTION) pending for injection. This means that...Quoted 2 lines
If so
Reviewed-by Jan Beulich <[email protected]>... the ICEBP-#DB-vectored
TASK_SWITCHwill now find %rip pointing after the ICEBP instruction, rather than at it, making it consistent with every other #DB-vectoredTASK_SWITCH.This does means that an early task-switch fault for ICEBP will reliably be delivered with the wrong (i.e. trap) semantics, but this is less bad than mixed fault/trap semantics depending on whether the source of the ICEBP was introspection/emulation or native execution.
Quoted 7 lines
We could restore proper fault behaviour by extending
svm_emul_swint_injection()to figure out that a task switch is needed, and invokehvm_task_switch()directly, but I don't have enough TUITS right now.Quoted 1 line
Otherwise I guess I'm still missing something.
I hope this clears it up.
Well, it helps, but you don't really answer the question: Is "trap" in that sentence of the description really correct? I.e. don't you instead mean "fault" there?
Quoted 8 lines
Quoted 7 lines
I've reworded that bit to:
Unconditionally intercept ICEBP. This does have NRIPs support as it is an instruction intercept, which allows us allows us to move %rip forwards appropriately before the
TASK_SWITCHintercept is hit. This allows...Any better?
Ah yes, thanks. (But please drop one of the two "allows us".)
Oops yes. Irritatingly, that causes #DB-vectored to move onto a new line, and trigger Git's comment syntax. I'll tweak a little bit more.
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 2/3] x86/svm: Always intercept ICEBP
In reply to Andrew CooperQuoted 23 lines · Andrew Cooper
ICEBP isn't handled well by SVM.
The VMexit state for a #DB-vectored
TASK_SWITCHhas %rip pointing to the appropriate instruction boundary (fault or trap, as appropriate), except for an ICEBP-induced #DBTASK_SWITCH, where %rip points at the ICEBP instruction rather than after it. As ICEBP isn't distinguished in the vectoring event type, the state is ambiguous.To add to the confusion, an ICEBP which occurs due to Introspection intercepting the instruction, or from
x86_emulate()will have %rip updated as a consequence of partial emulation required to inject an ICEBP event in the first place.We could in principle spot the non-injected case in the
TASK_SWITCHhandler, but this still results in complexity if the ICEBP instruction also has an Instruction Breakpoint active on it (which genuinely has fault semantics).Unconditionally intercept ICEBP. This does have a trap semantics for the intercept, and allows us to move %rip forwards appropriately before the
TASK_SWITCHintercept is hit. This makes the behaviour of #DB-vectored switches consistent however the ICEBP #DB came about, and avoids special cases in theTASK_SWITCHintercept.
Quoted 5 lines
This in turn allows for the removal of the conditional
hvm_set_icebp_interception()logic used by the monitor subsystem, as ICEBP's will now always be submitted for monitoring checks.Signed-off-by Andrew Cooper <[email protected]>
AFAICT this brings AMD implementation inline with Intel that also will unconditionally vmexit on icebp?
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 2/3] x86/svm: Always intercept ICEBP
Quoted 28 lines · Roger Pau Monné
Quoted 27 lines · Andrew Cooper
ICEBP isn't handled well by SVM.
The VMexit state for a #DB-vectored
TASK_SWITCHhas %rip pointing to the appropriate instruction boundary (fault or trap, as appropriate), except for an ICEBP-induced #DBTASK_SWITCH, where %rip points at the ICEBP instruction rather than after it. As ICEBP isn't distinguished in the vectoring event type, the state is ambiguous.To add to the confusion, an ICEBP which occurs due to Introspection intercepting the instruction, or from
x86_emulate()will have %rip updated as a consequence of partial emulation required to inject an ICEBP event in the first place.We could in principle spot the non-injected case in the
TASK_SWITCHhandler, but this still results in complexity if the ICEBP instruction also has an Instruction Breakpoint active on it (which genuinely has fault semantics).Unconditionally intercept ICEBP. This does have a trap semantics for the intercept, and allows us to move %rip forwards appropriately before the
TASK_SWITCHintercept is hit. This makes the behaviour of #DB-vectored switches consistent however the ICEBP #DB came about, and avoids special cases in theTASK_SWITCHintercept.This in turn allows for the removal of the conditional
hvm_set_icebp_interception()logic used by the monitor subsystem, as ICEBP's will now always be submitted for monitoring checks.
Quoted 5 lines
Quoted 1 line
Signed-off-by Andrew Cooper <[email protected]>Reviewed-by Roger Pau Monné <[email protected]>AFAICT this brings AMD implementation inline with Intel that also will unconditionally vmexit on icebp?
VT-x and SVM handle things quite differently.
VT-x has no instruction intercept for ICEBP, but the #DB intercept will triggered by an ICEBP instruction. ICEBP has its own event type (Privileged Software Exception, which is an amusing name considering it is an unprivleged instruction, bypasses privilege checks, and sets the External bit in an error code).
SVM does have an instruction intercept for ICEBP, but the #DB from ICEBP's don't trigger the normal #DB intercept. However, secondary #DB's generated by ICEBP's unintercepted #DB do trigger the #DB intercept.
For safety reasons we must intercept #DB to prevent CPU deadlocks. This means that ICEBP are in practice always intercepted on Intel due to their #DB side effect, but they weren't intercepted on AMD, which is why the monitor subsystem had a way of turning interception on.
So yes, the overall effect is that ICEBPs will now unconditionally vmexit on both Intel and AMD, but underlying mechanism for why they vmexit is still vendor-specific.
~Andrew
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 2/3] x86/svm: Always intercept ICEBP
Quoted 47 lines · Andrew Cooper
Quoted 28 lines · Roger Pau Monné
Quoted 27 lines · Andrew Cooper
ICEBP isn't handled well by SVM.
The VMexit state for a #DB-vectored
TASK_SWITCHhas %rip pointing to the appropriate instruction boundary (fault or trap, as appropriate), except for an ICEBP-induced #DBTASK_SWITCH, where %rip points at the ICEBP instruction rather than after it. As ICEBP isn't distinguished in the vectoring event type, the state is ambiguous.To add to the confusion, an ICEBP which occurs due to Introspection intercepting the instruction, or from
x86_emulate()will have %rip updated as a consequence of partial emulation required to inject an ICEBP event in the first place.We could in principle spot the non-injected case in the
TASK_SWITCHhandler, but this still results in complexity if the ICEBP instruction also has an Instruction Breakpoint active on it (which genuinely has fault semantics).Unconditionally intercept ICEBP. This does have a trap semantics for the intercept, and allows us to move %rip forwards appropriately before the
TASK_SWITCHintercept is hit. This makes the behaviour of #DB-vectored switches consistent however the ICEBP #DB came about, and avoids special cases in theTASK_SWITCHintercept.This in turn allows for the removal of the conditional
hvm_set_icebp_interception()logic used by the monitor subsystem, as ICEBP's will now always be submitted for monitoring checks.Quoted 5 lines
Quoted 1 line
Signed-off-by Andrew Cooper <[email protected]>Reviewed-by Roger Pau Monné <[email protected]>AFAICT this brings AMD implementation inline with Intel that also will unconditionally vmexit on icebp?
VT-x and SVM handle things quite differently.
VT-x has no instruction intercept for ICEBP, but the #DB intercept will triggered by an ICEBP instruction. ICEBP has its own event type (Privileged Software Exception, which is an amusing name considering it is an unprivleged instruction, bypasses privilege checks, and sets the External bit in an error code).
SVM does have an instruction intercept for ICEBP, but the #DB from ICEBP's don't trigger the normal #DB intercept. However, secondary #DB's generated by ICEBP's unintercepted #DB do trigger the #DB intercept.
Quoted 8 lines
For safety reasons we must intercept #DB to prevent CPU deadlocks. This means that ICEBP are in practice always intercepted on Intel due to their #DB side effect, but they weren't intercepted on AMD, which is why the monitor subsystem had a way of turning interception on.
So yes, the overall effect is that ICEBPs will now unconditionally vmexit on both Intel and AMD, but underlying mechanism for why they vmexit is still vendor-specific.
Thanks for the detailed explanation, I realized vmx didn't have a ICEBP specific VMEXIT reason, but I assumed the #DB would be trapped as that's how the monitor system intercepts those.
Roger.
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH v2 3/3 x86/svm: Write the correct %eip into the outgoing task
In reply to Andrew CooperThe TASK_SWITCH vmexit has fault semantics, and doesn't provide any NRIPs assistance with instruction length. As a result, any instruction-induced task switch has the outgoing task's %eip pointing at the instruction switch caused the switch, rather than after it.
This causes callers of task gates to livelock (repeatedly execute the call/jmp to enter the task), and any restartable task to become a nop after its first use (the (re)entry state points at the ret/iret used to exit the task).
32bit Windows in particular is known to use task gates for NMI handling, and to use NMI IPIs.
In the task switch handler, distinguish instruction-induced from interrupt/exception-induced task switches, and decode the instruction under %rip to calculate its length.
---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Juergen Gross <[email protected]>
v2:
* Correct ModRM calculation for Grp5.
* Never inject #GP from svm_get_task_switch_insn_len(). Dump emul ctxt and jump to crash_or_fault from the caller.
* Drop insn length for BOUND. It has fault semantics.
* Cope with HW_EXCEPTION #BP/#OF which do need instruction length calculations.
* Don't special case #DB
--- xen/arch/x86/hvm/svm/emulate.c | 54 ++++++++++++++++++++++++++++++++ xen/arch/x86/hvm/svm/svm.c | 58 +++++++++++++++++++++++++++++------ xen/include/asm-x86/hvm/svm/emulate.h | 1 + 3 files changed, 103 insertions(+), 10 deletions(-)
xen/arch/x86/hvm/svm/emulate.c +54 -0
@@ -117,6 +117,60 @@ unsigned int svm_get_insn_len(struct vcpu *v, unsigned int instr_enc) }/* +* TASK_SWITCH vmexits never provide an instruction length. We must always +* decode under %rip to find the answer. +*/ +unsigned int svm_get_task_switch_insn_len (void ) +{ +struct hvm_emulate_ctxt ctxt; +struct x86_emulate_state *state; +unsigned int emul_len, modrm_reg; + +hvm_emulate_init_once (&ctxt,NULL ,guest_cpu_user_regs ()); +hvm_emulate_init_per_insn (&ctxt,NULL ,0 ); + state =x86_decode_insn (&ctxt.ctxt, hvmemul_insn_fetch); +if (IS_ERR_OR_NULL (state) ) +return 0 ; + + emul_len =x86_insn_length (state, &ctxt.ctxt); + +/* +* Check for an instruction which can cause a task switch. Any far +* jmp/call/ret, any software interrupt/exception with trap semantics +* (except icebp - handled specially), and iret. +*/ +switch ( ctxt.ctxt.opcode ) + { +case 0xff :/* Grp 5 */ +/* call / jmp (far, absolute indirect) */ +if ( (unsigned int )x86_insn_modrm (state,NULL , &modrm_reg) >=3 || + (modrm_reg !=3 && modrm_reg !=5 ) ) + { +default : +printk (XENLOG_G_WARNING "Bad instruction for task switch\n" ); +hvm_dump_emulation_state (XENLOG_G_WARNING ,"SVM Insn len" , + &ctxt,X86EMUL_UNHANDLEABLE ); + emul_len =0 ; +break ; + } +/* Fallthrough */ +case 0x9a :/* call (far, absolute) */ +case 0xca :/* ret imm16 (far) */ +case 0xcb :/* ret (far) */ +case 0xcc :/* int3 */ +case 0xcd :/* int imm8 */ +case 0xce :/* into */ +case 0xcf :/* iret */ +case 0xea :/* jmp (far, absolute) */ +break ; + } + +x86_emulate_free_state (state); + +return emul_len; +} + +/* * Local variables: * mode: C * c-file-style: "BSD"
xen/arch/x86/hvm/svm/svm.c +48 -10
76 lines not on this page · raw message
xen/include/asm-x86/hvm/svm/emulate.h +1 -0
@@ -51,6 +51,7 @@struct vcpu ;unsigned int svm_get_insn_len (struct vcpu *v,unsigned int instr_enc); +unsigned int svm_get_task_switch_insn_len (void );#endif /* __ASM_X86_HVM_SVM_EMULATE_H__ */
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v2 3/3] x86/svm: Write the correct %eip into the outgoing task
Quoted 9 lines · Andrew Cooper
The
TASK_SWITCHvmexit has fault semantics, and doesn't provide any NRIPs assistance with instruction length. As a result, any instruction-induced task switch has the outgoing task's %eip pointing at the instruction switch caused the switch, rather than after it.This causes callers of task gates to livelock (repeatedly execute the call/jmp to enter the task), and any restartable task to become a nop after its first use (the (re)entry state points at the ret/iret used to exit the task).
Quoted 8 lines
32bit Windows in particular is known to use task gates for NMI handling, and to use NMI IPIs.
In the task switch handler, distinguish instruction-induced from interrupt/exception-induced task switches, and decode the instruction under %rip to calculate its length.
Signed-off-by Andrew Cooper <[email protected]>
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Quoted 8 lines · Andrew Cooper
These patches want backporting due to the severity of patch 2. They should therefore be considered for 4.13 at this point.
Andrew Cooper (3):
x86/vtx: Fix fault semantics for early task switch failures x86/svm: Always intercept ICEBP
x86/svm: Write the correct %eip into the outgoing task
Quoted 10 lines
xen/arch/x86/hvm/hvm.c | 4 +- xen/arch/x86/hvm/svm/emulate.c | 54 ++++++++++++++++++++++++ xen/arch/x86/hvm/svm/svm.c | 77 ++++++++++++++++++++++------------- xen/arch/x86/hvm/svm/vmcb.c | 2 +- xen/arch/x86/hvm/vmx/vmx.c | 4 +- xen/arch/x86/monitor.c | 3 -- xen/include/asm-x86/hvm/hvm.h | 13 +----- xen/include/asm-x86/hvm/svm/emulate.h | 1 + 8 files changed, 109 insertions(+), 49 deletions(-)
For the series:
Juergen
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel