Skip to contentSkip to conversation

Fixes to debugging facilities

PATCH Superseded Jinoh Kang 2 participants 13 messages Aug 21, 2023 – Aug 22, 2023 1 message not in the archive

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Jinoh Kang

author
#0 lore raw

cover Fixes to debugging facilities

This is a rebased version of Andrew Cooper's debugging facilities patch:
https://lore.kernel.org/xen-devel/[email protected]/

Quoted 8 lines

So this started as a small fix for the vmentry failure (penultimate patch), and has snowballed...

I'm fairly confident that everything involving DEBUGCTL.BTF is broken, and there are definitely bugs with configuring DEBUGCTL.RTM (which really isn't helped by the fact that the GCC TSX intrinsics render the resulting code un-debuggable.) I'll defer fixing these swamps for now.

Quoted 8 lines

The first 4 patches probably want backporting to the stable trees, so I've taken care to move them ahead of patch 6 for backport reasons. While all fixes would ideally be backported, I can't find a way of fixing %dr6 merging (as it needs to be done precicely once) without a behavioural change in the monitor subsystem.

Patch 8 probably breaks introspection, so can't be taken at this point. See that patch for discussion of the problem and my best guess at a solution.

6 out of 11 patches from the 2018 patch series above, including the vmentry failure fix, have already been committed. This covers the remaining 5 patches.

One particular bug that the patch series fixes involves simultaneous hardware breakpoint exception and single-stepping exception occurring at the same PC (IP). Xen blindly sets singlestep (DR6.BS := 1) in this case, which interferes with userland debugging and allows (otherwise restricted) usermode programs to detect Xen HVM (or PVH). The following Linux x86-64 program demonstrates the bug:

-----------------------------------8<-----------------------------------

#include <stddef.h>
#include <stdlib.h>
#include <assert.h>
#include <unistd.h>
#include <signal.h>
#include <sys/ptrace.h>
#include <sys/wait.h>
#include <sys/user.h>
#include <stdio.h>

#define ABORT_ON_ERR(x) if ((x) == -1) abort();

int main(void)

{
    unsigned long cur_rip, cur_eflags, cur_dr6;
    int wstatus, exit_code;
    pid_t pid;

    ABORT_ON_ERR(pid = fork());
    if (pid == 0) {
        ABORT_ON_ERR(ptrace(PTRACE_TRACEME, 0, NULL, NULL));
        ABORT_ON_ERR(raise(SIGSTOP));
        _exit(0);
    }

    /* Wait for first ptrace event */
    if (waitpid(pid, &wstatus, 0) != pid) abort();
    if (!WIFSTOPPED(wstatus)) abort();

    /* Obtain current RIP value and perform sanity check */
    cur_rip = ptrace(PTRACE_PEEKUSER, pid, (void *)offsetof(struct user, regs.rip), &cur_rip);
    cur_dr6 = ptrace(PTRACE_PEEKUSER, pid, (void *)offsetof(struct user, u_debugreg[6]), &cur_dr6);
    assert(cur_dr6 == 0xffff0ff0UL);

    /* Set up debug registers and set EFLAGS.TF */
    cur_eflags = ptrace(PTRACE_PEEKUSER, pid, (void *)offsetof(struct user, regs.eflags), &cur_eflags);
    ABORT_ON_ERR(ptrace(PTRACE_POKEUSER, pid, (void *)offsetof(struct user, regs.eflags), (void *)(cur_eflags | 0x100UL)));
    ABORT_ON_ERR(ptrace(PTRACE_POKEUSER, pid, (void *)offsetof(struct user, u_debugreg[0]), (void *)cur_rip));
    ABORT_ON_ERR(ptrace(PTRACE_POKEUSER, pid, (void *)offsetof(struct user, u_debugreg[7]), (void *)1L));

    /* Continue execution to trigger hardware breakpoint */
    ABORT_ON_ERR(ptrace(PTRACE_CONT, pid, NULL, (unsigned long)0));
    if (waitpid(pid, &wstatus, 0) != pid) abort();
    if (!(WIFSTOPPED(wstatus) && WSTOPSIG(wstatus) == SIGTRAP)) abort();

    /* Detect if Xen has tampered with DR6 */
    cur_dr6 = ptrace(PTRACE_PEEKUSER, pid, (void *)offsetof(struct user, u_debugreg[6]), &cur_dr6);
    fprintf(stderr, "DR6 = 0x%08lx\n", cur_dr6);
    if (cur_dr6 == 0xffff0ff1UL)
    {
        fputs("Running on bare-metal, Xen PV, or non-Xen VMM\n", stdout);
        exit_code = EXIT_FAILURE;
    }
    else
    {
        fputs("Running on Xen HVM\n", stdout);
        exit_code = EXIT_SUCCESS;
    }

    /* Tear down debug registers and unset EFLAGS.TF */
    cur_eflags = ptrace(PTRACE_PEEKUSER, pid, (void *)offsetof(struct user, regs.eflags), &cur_eflags);
    ABORT_ON_ERR(ptrace(PTRACE_POKEUSER, pid, (void *)offsetof(struct user, regs.eflags), (void *)(cur_eflags & ~0x100UL)));
    ABORT_ON_ERR(ptrace(PTRACE_POKEUSER, pid, (void *)offsetof(struct user, u_debugreg[7]), (void *)0L));

    /* Continue execution to let child process exit */
    ABORT_ON_ERR(ptrace(PTRACE_CONT, pid, NULL, (unsigned long)0));
    if (waitpid(pid, &wstatus, 0) != pid) abort();
    if (!(WIFEXITED(wstatus) && WEXITSTATUS(wstatus) == 0)) abort();

    return exit_code;
}

-----------------------------------8<-----------------------------------

Andrew Cooper (5):
x86: Fix calculation of %dr6/7 reserved bits
x86/emul: Add pending_dbg field to x86_event
x86/hvm: RFC - PROBABLY BROKEN - Defer all debugging/monitor actions

    to {svm,vmx}_inject_event()

x86: Fix merging of new status bits into %dr6
x86/dbg: Cleanup of legacy dr6 constants

 xen/arch/x86/domain.c                  |   7 +-
 xen/arch/x86/hvm/emulate.c             |   3 +-
 xen/arch/x86/hvm/hvm.c                 |   8 +-
 xen/arch/x86/hvm/svm/svm.c             | 126 ++++++++++++++-----------
 xen/arch/x86/hvm/vmx/vmx.c             | 112 ++++++++++------------
 xen/arch/x86/include/asm/debugreg.h    |  94 +++++++++++++-----
 xen/arch/x86/include/asm/domain.h      |  12 +++
 xen/arch/x86/include/asm/hvm/hvm.h     |  15 ++-
 xen/arch/x86/include/asm/x86-defns.h   |  10 --
 xen/arch/x86/mm/shadow/multi.c         |   5 +-
 xen/arch/x86/pv/emul-priv-op.c         |  13 ++-
 xen/arch/x86/pv/emulate.c              |   6 +-
 xen/arch/x86/pv/misc-hypercalls.c      |  16 +---
 xen/arch/x86/pv/ro-page-fault.c        |   3 +-
 xen/arch/x86/pv/traps.c                |  17 +++-
 xen/arch/x86/traps.c                   |   8 +-
 xen/arch/x86/x86_emulate/x86_emulate.h |   5 +-
 17 files changed, 262 insertions(+), 198 deletions(-)
Signature · 1 line
2.41.0
author
#1 lore raw

PATCH 1/5 x86: Fix calculation of %dr6/7 reserved bits

From: Andrew Cooper <[email protected]>

The reserved bit calculations for %dr6 and %dr7 depend on whether the VM has the Restricted Transnational Memory feature available.

Introduce adjust_dr{6,7}_rsvd() and replace the opencoded logic and constants (except for DR_STATUS_RESERVED_ONE which is (mis)used elsewhere and will be removed after future bugfixes). The use of these helpers in set_debugreg() covers toolstack values for PV guests, but HVM guests need similar treatment.

The use of the guests cpuid policy is less than optimal in the create/restore paths. However in such cases, the policy will be the guest maximum policy, which will be more permissive with respect to the RTM feature.

---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>

---
 xen/arch/x86/domain.c               |  7 +++--
 xen/arch/x86/hvm/hvm.c              |  6 ++--
 xen/arch/x86/include/asm/debugreg.h | 44 +++++++++++++++++++++++++----
 xen/arch/x86/pv/misc-hypercalls.c   | 16 +++--------
 4 files changed, 50 insertions(+), 23 deletions(-)
xen/arch/x86/domain.c +4 -3
index fe86a7f853..a39710b5af 100644
@@ -1053,6 +1053,7 @@ int arch_set_info_guest(
     struct vcpu *v, vcpu_guest_context_u c)
 {
     struct domain *d = v->domain;
+    const struct cpu_policy *cp = d->arch.cpuid;
     unsigned int i;
     unsigned long flags;
     bool compat;
@@ -1165,10 +1166,10 @@ int arch_set_info_guest(
 
     if ( is_hvm_domain(d) )
     {
-        for ( i = 0; i < ARRAY_SIZE(v->arch.dr); ++i )
+        for ( i = 0; i < ARRAY_SIZE(v->arch.dr) - 2; ++i )
             v->arch.dr[i] = c(debugreg[i]);
-        v->arch.dr6 = c(debugreg[6]);
-        v->arch.dr7 = c(debugreg[7]);
+        v->arch.dr6 = adjust_dr6_rsvd(c(debugreg[6]), cp->feat.rtm);
+        v->arch.dr7 = adjust_dr7_rsvd(c(debugreg[7]), cp->feat.rtm);
 
         if ( v->vcpu_id == 0 )
             d->vm_assist = c.nat->vm_assist;
xen/arch/x86/hvm/hvm.c +4 -2
index 3a99c0ff20..66ead0b878 100644
@@ -33,6 +33,7 @@
 #include <asm/shadow.h>
 #include <asm/hap.h>
 #include <asm/current.h>
+#include <asm/debugreg.h>
 #include <asm/e820.h>
 #include <asm/io.h>
 #include <asm/regs.h>
@@ -985,6 +986,7 @@ unsigned long hvm_cr4_guest_valid_bits(const struct domain *d)
 
 static int cf_check hvm_load_cpu_ctxt(struct domain *d, hvm_domain_context_t *h)
 {
+    const struct cpu_policy *cp = d->arch.cpuid;
     unsigned int vcpuid = hvm_load_instance(h);
     struct vcpu *v;
     struct hvm_hw_cpu ctxt;
@@ -1166,8 +1168,8 @@ static int cf_check hvm_load_cpu_ctxt(struct domain *d, hvm_domain_context_t *h)
     v->arch.dr[1] = ctxt.dr1;
     v->arch.dr[2] = ctxt.dr2;
     v->arch.dr[3] = ctxt.dr3;
-    v->arch.dr6   = ctxt.dr6;
-    v->arch.dr7   = ctxt.dr7;
+    v->arch.dr6   = adjust_dr6_rsvd(ctxt.dr6, cp->feat.rtm);
+    v->arch.dr7   = adjust_dr7_rsvd(ctxt.dr7, cp->feat.rtm);
 
     hvmemul_cancel(v);
 
xen/arch/x86/include/asm/debugreg.h +38 -6

66 lines not on this page · raw message

xen/arch/x86/pv/misc-hypercalls.c +4 -12
index b11bd718b7..e44f2556c8 100644
@@ -56,6 +56,7 @@ long do_fpu_taskswitch(int set)
 long set_debugreg(struct vcpu *v, unsigned int reg, unsigned long value)
 {
     struct vcpu *curr = current;
+    const struct cpu_policy *cp = curr->domain->arch.cpuid;
 
     switch ( reg )
     {
@@ -86,12 +87,7 @@ long set_debugreg(struct vcpu *v, unsigned int reg, unsigned long value)
         if ( value != (uint32_t)value )
             return -EINVAL;
 
-        /*
-         * DR6: Bits 4-11,16-31 reserved (set to 1).
-         *      Bit 12 reserved (set to 0).
-         */
-        value &= ~DR_STATUS_RESERVED_ZERO; /* reserved bits => 0 */
-        value |=  DR_STATUS_RESERVED_ONE;  /* reserved bits => 1 */
+        value = adjust_dr6_rsvd(value, cp->feat.rtm);
 
         v->arch.dr6 = value;
         if ( v == curr )
@@ -108,12 +104,8 @@ long set_debugreg(struct vcpu *v, unsigned int reg, unsigned long value)
         if ( value != (uint32_t)value )
             return -EINVAL;
 
-        /*
-         * DR7: Bit 10 reserved (set to 1).
-         *      Bits 11-12,14-15 reserved (set to 0).
-         */
-        value &= ~DR_CONTROL_RESERVED_ZERO; /* reserved bits => 0 */
-        value |=  DR_CONTROL_RESERVED_ONE;  /* reserved bits => 1 */
+        value = adjust_dr7_rsvd(value, cp->feat.rtm);
+
         /*
          * Privileged bits:
          *      GD (bit 13): must be 0.
Signature · 1 line
2.41.0
author
#2 lore raw

Re: [PATCH 1/5] x86: Fix calculation of %dr6/7 reserved bits

Quoted 4 lines · Jinoh Kang

From: Andrew Cooper <[email protected]>

The reserved bit calculations for %dr6 and %dr7 depend on whether the VM has the Restricted Transnational Memory feature available.

s/Transnational/Transactional/.

It was in the original review, but I missed the change. Apologies.

Signature · 2 lines
Sincerely,
Jinoh Kang
#3 lore raw

Re: [PATCH 1/5] x86: Fix calculation of %dr6/7 reserved bits

Quoted 9 lines · Jinoh Kang
Quoted 4 lines · Jinoh Kang

From: Andrew Cooper <[email protected]>

The reserved bit calculations for %dr6 and %dr7 depend on whether the VM has the Restricted Transnational Memory feature available.

s/Transnational/Transactional/.

It was in the original review, but I missed the change. Apologies.

But that's not the only change that was requested back then. There was one aspect Andrew didn't like, so leaving that part as is would be fine. But for the items he didn't further respond to, I'd expect a re-submission to take care of them. Plus, if you didn't address anything, you would want to (a) mention that and (b) take Roger's R-b that was provided at the time (unless of course re-basing was resulting in massive changes).

As I expect the same to be the case for the other patches, I'm not sure it's worth looking at them (again).

Jan

author
#4 lore raw

Re: [PATCH 1/5] x86: Fix calculation of %dr6/7 reserved bits

Quoted 9 lines · Jan Beulich
Quoted 8 lines · Jinoh Kang
Quoted 4 lines · Jinoh Kang

From: Andrew Cooper <[email protected]>

The reserved bit calculations for %dr6 and %dr7 depend on whether the VM has the Restricted Transnational Memory feature available.

s/Transnational/Transactional/.

Quoted 7 lines
Quoted 1 line

It was in the original review, but I missed the change. Apologies.

But that's not the only change that was requested back then. There was one aspect Andrew didn't like, so leaving that part as is would be fine. But for the items he didn't further respond to, I'd expect a re-submission to take care of them. Plus, if you didn't address anything, you would want to (a) mention that

Right. I'll try to address the rest and provide rationale for those that aren't feasible.

Quoted 3 lines

and (b) take Roger's R-b
that was provided at the time (unless of course re-basing was resulting in massive changes).

There were many changes, some arguably cosmetic and some not-so-syntactic ones. Quite a long time has been passed, so I was unsure if the R-b's held up to the present moment.

Quoted 3 lines

As I expect the same to be the case for the other patches, I'm not sure it's worth looking at them (again).

ACK. Please consider this patchset withdrawn.

Quoted 2 lines

Jan

Signature · 2 lines
Sincerely,
Jinoh Kang
author
#5 lore raw

Re: [PATCH 1/5] x86: Fix calculation of %dr6/7 reserved bits

In reply to Jan Beulich
Quoted 4 lines · Jan Beulich

But that's not the only change that was requested back then. There was one aspect Andrew didn't like, so leaving that part as is would be fine. But for the items he didn't further respond to, I'd expect a re-submission to take care of them.

Somehow I assumed that Andrew's branch had already contained the relevant updates, which was clearly wrong and was a sloppy thinking on my part. Also, rushing to submission didn't _really_ help.

Again, apologies for wasting your time.

Signature · 2 lines
Sincerely,
Jinoh Kang
author
#6 lore raw

PATCH 2/5 x86/emul: Add pending_dbg field to x86_event

In reply to Jinoh Kang

From: Andrew Cooper <[email protected]>

All #DB exceptions result in an update of %dr6, but this isn't captured in Xen's handling.

PV guests generally work by modifying %dr6 before raising #DB, whereas HVM guests do nothing and have a single-step special case in the lowest levels of {vmx,svm}_inject_event(). All of this is buggy, but in particular, task switches with the trace flag never end up signalling BT in %dr6.

To begin resolving this issue, add a new pending_dbg field to x86_event (unioned with cr2 to avoid taking any extra space), and introduce {pv,hvm}_inject_debug_exn() helpers to replace the current callers using {pv,hvm}_inject_hw_exception().

A key property is that pending_dbg is taken with positive polarity to deal with RTM sensibly. Most callers pass in a constant, but callers passing in a hardware %dr6 value need to xor the value with X86_DR6_DEFAULT to flip the polarity of RTM and reserved fields.

For PV guests, move the ad-hoc updating of %dr6 into pv_inject_event(). This in principle breaks the handing of RTM in do_debug(), but PV guests can't actually enable MSR_DEBUGCTL.RTM yet, so this doesn't matter in practice.

---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jun Nakajima <[email protected]>
CC: Kevin Tian <[email protected]>
CC: Tim Deegan <[email protected]>

---
 xen/arch/x86/hvm/emulate.c             |  3 ++-
 xen/arch/x86/hvm/hvm.c                 |  2 +-
 xen/arch/x86/hvm/svm/svm.c             |  9 ++++++---
 xen/arch/x86/hvm/vmx/vmx.c             | 13 ++++++++-----
 xen/arch/x86/include/asm/debugreg.h    |  3 +++
 xen/arch/x86/include/asm/domain.h      | 12 ++++++++++++
 xen/arch/x86/include/asm/hvm/hvm.h     | 15 ++++++++++++++-
 xen/arch/x86/mm/shadow/multi.c         |  5 +++--
 xen/arch/x86/pv/emul-priv-op.c         | 11 +++++------
 xen/arch/x86/pv/emulate.c              |  6 ++----
 xen/arch/x86/pv/ro-page-fault.c        |  3 ++-
 xen/arch/x86/pv/traps.c                | 16 ++++++++++++----
 xen/arch/x86/traps.c                   |  6 +-----
 xen/arch/x86/x86_emulate/x86_emulate.h |  5 ++++-
 14 files changed, 75 insertions(+), 34 deletions(-)
xen/arch/x86/hvm/emulate.c +2 -1
index 9b6e4c8bc6..129403ad90 100644
@@ -16,6 +16,7 @@
 #include <xen/paging.h>
 #include <xen/trace.h>
 #include <xen/vm_event.h>
+#include <asm/debugreg.h>
 #include <asm/event.h>
 #include <asm/i387.h>
 #include <asm/xstate.h>
@@ -2673,7 +2674,7 @@ static int _hvm_emulate_one(struct hvm_emulate_ctxt *hvmemul_ctxt,
     }
 
     if ( hvmemul_ctxt->ctxt.retire.singlestep )
-        hvm_inject_hw_exception(X86_EXC_DB, X86_EVENT_NO_EC);
+        hvm_inject_debug_exn(X86_DR6_BS);
 
     new_intr_shadow = hvmemul_ctxt->intr_shadow;
 
xen/arch/x86/hvm/hvm.c +1 -1
index 66ead0b878..c3d46841c2 100644
@@ -3229,7 +3229,7 @@ void hvm_task_switch(
     }
 
     if ( (tss.trace & 1) && !exn_raised )
-        hvm_inject_hw_exception(X86_EXC_DB, X86_EVENT_NO_EC);
+        hvm_inject_debug_exn(X86_DR6_BT);
 
  out:
     hvm_unmap_entry(optss_desc);
xen/arch/x86/hvm/svm/svm.c +6 -3
index 01dd592d9b..ac3123c56f 100644
@@ -96,7 +96,7 @@ void __update_guest_eip(struct cpu_user_regs *regs, unsigned int inst_len)
     curr->arch.hvm.svm.vmcb->int_stat.intr_shadow = 0;
 
     if ( regs->eflags & X86_EFLAGS_TF )
-        hvm_inject_hw_exception(X86_EXC_DB, X86_EVENT_NO_EC);
+        hvm_inject_debug_exn(X86_DR6_BS);
 }
 
 static void cf_check svm_cpu_down(void)
@@ -2755,7 +2755,10 @@ void svm_vmexit_handler(void)
                 goto unexpected_exit_type;
             if ( !rc )
                 hvm_inject_exception(X86_EXC_DB,
-                                     trap_type, insn_len, X86_EVENT_NO_EC);
+                                     trap_type, insn_len, X86_EVENT_NO_EC,
+                                     exit_reason == VMEXIT_ICEBP ? 0 :
+                                     /* #DB - Hardware already updated dr6. */
+                                     vmcb_get_dr6(vmcb) ^ X86_DR6_DEFAULT);
         }
         else
             domain_pause_for_debugger();
@@ -2785,7 +2788,7 @@ void svm_vmexit_handler(void)
            if ( !rc )
                hvm_inject_exception(X86_EXC_BP,
                                     X86_EVENTTYPE_SW_EXCEPTION,
-                                    insn_len, X86_EVENT_NO_EC);
+                                    insn_len, X86_EVENT_NO_EC, 0 /* N/A */);
         }
         break;
 
xen/arch/x86/hvm/vmx/vmx.c +8 -5
index 7ec44018d4..716115332b 100644
@@ -3068,7 +3068,7 @@ void update_guest_eip(void)
     }
 
     if ( regs->eflags & X86_EFLAGS_TF )
-        hvm_inject_hw_exception(X86_EXC_DB, X86_EVENT_NO_EC);
+        hvm_inject_debug_exn(X86_DR6_BS);
 }
 
 static void cf_check vmx_fpu_dirty_intercept(void)
@@ -3911,7 +3911,7 @@ static int vmx_handle_eoi_write(void)
  * It is the callers responsibility to ensure that this function is only used
  * in the context of an appropriate vmexit.
  */
-static void vmx_propagate_intr(unsigned long intr)
+static void vmx_propagate_intr(unsigned long intr, unsigned long pending_dbg)
 {
     struct x86_event event = {
         .vector = MASK_EXTR(intr, INTR_INFO_VECTOR_MASK),
@@ -3935,6 +3935,9 @@ static void vmx_propagate_intr(unsigned long intr)
     else
         event.insn_len = 0;
 
+    if ( event.vector == X86_EXC_DB )
+        event.pending_dbg = pending_dbg;
+
     hvm_inject_event(&event);
 }
 
@@ -4300,7 +4303,7 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
                 if ( rc < 0 )
                     goto exit_and_crash;
                 if ( !rc )
-                    vmx_propagate_intr(intr_info);
+                    vmx_propagate_intr(intr_info, exit_qualification);
             }
             else
                 domain_pause_for_debugger();
@@ -4321,7 +4324,7 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
                 if ( rc < 0 )
                     goto exit_and_crash;
                 if ( !rc )
-                    vmx_propagate_intr(intr_info);
+                    vmx_propagate_intr(intr_info, 0 /* N/A */);
             }
             else
             {
@@ -4361,7 +4364,7 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
             break;
         case X86_EXC_AC:
             HVMTRACE_1D(TRAP, vector);
-            vmx_propagate_intr(intr_info);
+            vmx_propagate_intr(intr_info, 0 /* N/A */);
             break;
         case X86_EXC_NMI:
             if ( MASK_EXTR(intr_info, INTR_INFO_INTR_TYPE_MASK) !=
xen/arch/x86/include/asm/debugreg.h +3 -0
index 8be60910b4..a1df74c02e 100644
@@ -83,6 +83,9 @@
     asm volatile ( "mov %%db" #reg ",%0" : "=r" (__val) );  \
     __val;                                                  \
 })
+
+struct vcpu;
+
 long set_debugreg(struct vcpu *, unsigned int reg, unsigned long value);
 void activate_debugregs(const struct vcpu *);
 
xen/arch/x86/include/asm/domain.h +12 -0
index c2d9fc333b..eba11adf33 100644
@@ -729,6 +729,18 @@ static inline void pv_inject_hw_exception(unsigned int vector, int errcode)
     pv_inject_event(&event);
 }
 
+static inline void pv_inject_debug_exn(unsigned long pending_dbg)
+{
+    struct x86_event event = {
+        .vector      = X86_EXC_DB,
+        .type        = X86_EVENTTYPE_HW_EXCEPTION,
+        .error_code  = X86_EVENT_NO_EC,
+        .pending_dbg = pending_dbg,
+    };
+
+    pv_inject_event(&event);
+}
+
 static inline void pv_inject_page_fault(int errcode, unsigned long cr2)
 {
     const struct x86_event event = {
xen/arch/x86/include/asm/hvm/hvm.h +14 -1
index 6d53713fc3..43989f1681 100644
@@ -503,13 +503,14 @@ hvm_get_cpl(struct vcpu *v)
 
 static inline void hvm_inject_exception(
     unsigned int vector, unsigned int type,
-    unsigned int insn_len, int error_code)
+    unsigned int insn_len, int error_code, unsigned long extra)
 {
     struct x86_event event = {
         .vector = vector,
         .type = type,
         .insn_len = insn_len,
         .error_code = error_code,
+        .cr2 = extra, /* Any union field will do. */
     };
 
     hvm_inject_event(&event);
@@ -526,6 +527,18 @@ static inline void hvm_inject_hw_exception(unsigned int vector, int errcode)
     hvm_inject_event(&event);
 }
 
+static inline void hvm_inject_debug_exn(unsigned long pending_dbg)
+{
+    struct x86_event event = {
+        .vector      = X86_EXC_DB,
+        .type        = X86_EVENTTYPE_HW_EXCEPTION,
+        .error_code  = X86_EVENT_NO_EC,
+        .pending_dbg = pending_dbg,
+    };
+
+    hvm_inject_event(&event);
+}
+
 static inline void hvm_inject_page_fault(int errcode, unsigned long cr2)
 {
     struct x86_event event = {
xen/arch/x86/mm/shadow/multi.c +3 -2
index cf74fdf5dd..6056626912 100644
@@ -15,6 +15,7 @@
 #include <xen/perfc.h>
 #include <xen/domain_page.h>
 #include <xen/iocap.h>
+#include <asm/debugreg.h>
 #include <xsm/xsm.h>
 #include <asm/page.h>
 #include <asm/current.h>
@@ -2788,7 +2789,7 @@ static int cf_check sh_page_fault(
 #endif
 
     if ( emul_ctxt.ctxt.retire.singlestep )
-        hvm_inject_hw_exception(X86_EXC_DB, X86_EVENT_NO_EC);
+        hvm_inject_debug_exn(X86_DR6_BS);
 
 #if GUEST_PAGING_LEVELS == 3 /* PAE guest */
     /*
@@ -2829,7 +2830,7 @@ static int cf_check sh_page_fault(
                 TRACE_SHADOW_PATH_FLAG(TRCE_SFLAG_EMULATION_LAST_FAILED);
 
                 if ( emul_ctxt.ctxt.retire.singlestep )
-                    hvm_inject_hw_exception(X86_EXC_DB, X86_EVENT_NO_EC);
+                    hvm_inject_debug_exn(X86_DR6_BS);
 
                 break; /* Don't emulate again if we failed! */
             }
xen/arch/x86/pv/emul-priv-op.c +5 -6
index 142bc4818c..72d0514e74 100644
@@ -1360,12 +1360,11 @@ int pv_emulate_privileged_op(struct cpu_user_regs *regs)
     case X86EMUL_OKAY:
         if ( ctxt.ctxt.retire.singlestep )
             ctxt.bpmatch |= DR_STEP;
-        if ( ctxt.bpmatch )
-        {
-            curr->arch.dr6 |= ctxt.bpmatch | DR_STATUS_RESERVED_ONE;
-            if ( !(curr->arch.pv.trap_bounce.flags & TBF_EXCEPTION) )
-                pv_inject_hw_exception(X86_EXC_DB, X86_EVENT_NO_EC);
-        }
+
+        if ( ctxt.bpmatch &&
+             !(curr->arch.pv.trap_bounce.flags & TBF_EXCEPTION) )
+            pv_inject_debug_exn(ctxt.bpmatch);
+
         /* fall through */
     case X86EMUL_RETRY:
         return EXCRET_fault_fixed;
xen/arch/x86/pv/emulate.c +2 -4
index e7a1c0a2cc..aa8af96c30 100644
@@ -71,11 +71,9 @@ void pv_emul_instruction_done(struct cpu_user_regs *regs, unsigned long rip)
 {
     regs->rip = rip;
     regs->eflags &= ~X86_EFLAGS_RF;
+
     if ( regs->eflags & X86_EFLAGS_TF )
-    {
-        current->arch.dr6 |= DR_STEP | DR_STATUS_RESERVED_ONE;
-        pv_inject_hw_exception(X86_EXC_DB, X86_EVENT_NO_EC);
-    }
+        pv_inject_debug_exn(X86_DR6_BS);
 }
 
 uint64_t pv_get_reg(struct vcpu *v, unsigned int reg)
xen/arch/x86/pv/ro-page-fault.c +2 -1
index cad28ef928..50c37fbd25 100644
@@ -9,6 +9,7 @@
  */
 
 #include <asm/pv/trace.h>
+#include <asm/debugreg.h>
 #include <asm/shadow.h>
 
 #include "emulate.h"
@@ -390,7 +391,7 @@ int pv_ro_page_fault(unsigned long addr, struct cpu_user_regs *regs)
         /* Fallthrough */
     case X86EMUL_OKAY:
         if ( ctxt.retire.singlestep )
-            pv_inject_hw_exception(X86_EXC_DB, X86_EVENT_NO_EC);
+            pv_inject_debug_exn(X86_DR6_BS);
 
         /* Fallthrough */
     case X86EMUL_RETRY:
xen/arch/x86/pv/traps.c +12 -4
index 74f333da7e..4f5641a47c 100644
@@ -13,6 +13,7 @@
 #include <xen/softirq.h>
 
 #include <asm/pv/trace.h>
+#include <asm/debugreg.h>
 #include <asm/shared.h>
 #include <asm/traps.h>
 #include <irq_vectors.h>
@@ -50,9 +51,9 @@ void pv_inject_event(const struct x86_event *event)
     tb->cs    = ti->cs;
     tb->eip   = ti->address;
 
-    if ( event->type == X86_EVENTTYPE_HW_EXCEPTION &&
-         vector == X86_EXC_PF )
+    switch ( vector | -(event->type == X86_EVENTTYPE_SW_INTERRUPT) )
     {
+    case X86_EXC_PF:
         curr->arch.pv.ctrlreg[2] = event->cr2;
         arch_set_cr2(curr, event->cr2);
 
@@ -62,9 +63,16 @@ void pv_inject_event(const struct x86_event *event)
             error_code |= PFEC_user_mode;
 
         trace_pv_page_fault(event->cr2, error_code);
-    }
-    else
+        break;
+
+    case X86_EXC_DB:
+        curr->arch.dr6 |= event->pending_dbg;
+        /* Fallthrough */
+
+    default:
         trace_pv_trap(vector, regs->rip, use_error_code, error_code);
+        break;
+    }
 
     if ( use_error_code )
     {
xen/arch/x86/traps.c +1 -5
index a898e1f2d7..640b60e0e5 100644
@@ -1989,17 +1989,13 @@ void do_debug(struct cpu_user_regs *regs)
         return;
     }
 
-    /* Save debug status register where guest OS can peek at it */
-    v->arch.dr6 |= (dr6 & ~X86_DR6_DEFAULT);
-    v->arch.dr6 &= (dr6 | ~X86_DR6_DEFAULT);
-
     if ( guest_kernel_mode(v, regs) && v->domain->debugger_attached )
     {
         domain_pause_for_debugger();
         return;
     }
 
-    pv_inject_hw_exception(X86_EXC_DB, X86_EVENT_NO_EC);
+    pv_inject_debug_exn(dr6 ^ X86_DR6_DEFAULT);
 }
 
 void do_entry_CP(struct cpu_user_regs *regs)
xen/arch/x86/x86_emulate/x86_emulate.h +4 -1
index 698750267a..e348e3c1d3 100644
@@ -78,7 +78,10 @@ struct x86_event {
     uint8_t       type;         /* X86_EVENTTYPE_* */
     uint8_t       insn_len;     /* Instruction length */
     int32_t       error_code;   /* X86_EVENT_NO_EC if n/a */
-    unsigned long cr2;          /* Only for X86_EXC_PF h/w exception */
+    union {
+        unsigned long cr2;         /* #PF */
+        unsigned long pending_dbg; /* #DB (new DR6 bits, positive polarity) */
+    };
 };
 
 /*
Signature · 1 line
2.41.0
author
#7 lore raw

PATCH 3/5 x86/hvm: RFC - PROBABLY BROKEN - Defer all debugging/monitor actions to {svm,vmx}_inject_event()

In reply to Jinoh Kang

From: Andrew Cooper <[email protected]>

Currently, there is a lot of functionality in the #DB intercepts, and some repeated functionality in the *_inject_event() logic.

The gdbsx code is implemented at both levels (albeit differently for #BP, which is presumably due to the fact that the old emulator behaviour used to be to move %rip forwards for traps), while the monitor behaviour only exists at the intercept level.

Updating of %dr6 is implemented (buggily) at both levels, but having it at both levels is problematic to implement correctly.

Rearrange the logic to have nothing interesting at the intercept level, and everything implemented at the injection level. Amongst other things, this means that the monitor subsystem will pick up debug actions from emulated events.

---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jun Nakajima <[email protected]>
CC: Kevin Tian <[email protected]>
CC: Razvan Cojocaru <[email protected]>
CC: Tamas K Lengyel <[email protected]>

This is RFC because it probably breaks introspection, as injection replies from the introspection engine will (probably, but I haven't confirmed) trigger new monitor events.

First of all, monitoring emulated debug events is a change in behaviour, although IMO it is more of a bugfix than a new feature. Also, similar changes will happen to other monitored events as we try to unify the intercept/emulation paths.

As for the recursive triggering of monitor events, I was considering extending the monitor infrastructure to have a "in monitor reply" boolean which causes hvm_monitor_debug() to ignore the recursive request.

Does this plan sound ok, or have I missed something more subtle with monitor handling?

---
 xen/arch/x86/hvm/svm/svm.c | 126 ++++++++++++++++++++-----------------
 xen/arch/x86/hvm/vmx/vmx.c | 102 +++++++++++++-----------------
 2 files changed, 110 insertions(+), 118 deletions(-)
xen/arch/x86/hvm/svm/svm.c +68 -58
index ac3123c56f..bd3adde0ec 100644
@@ -1328,19 +1328,50 @@ static void cf_check svm_inject_event(const struct x86_event *event)
     switch ( _event.vector | -(_event.type == X86_EVENTTYPE_SW_INTERRUPT) )
     {
     case X86_EXC_DB:
-        if ( regs->eflags & X86_EFLAGS_TF )
-        {
-            __restore_debug_registers(vmcb, curr);
-            vmcb_set_dr6(vmcb, vmcb_get_dr6(vmcb) | DR_STEP);
-        }
+        /*
+         * On AMD hardware, a #DB exception:
+         *  1) Merges new status bits into %dr6
+         *  2) Clears %dr7.gd and MSR_DEBUGCTL.{LBR,BTF}
+         *
+         * Item 1 is done by hardware before a #DB intercepted vmexit, but we
+         * may end up here from emulation so have to repeat it ourselves.
+         * Item 2 is done by hardware when injecting a #DB exception.
+         */
+        __restore_debug_registers(vmcb, curr);
+        vmcb_set_dr6(vmcb, vmcb_get_dr6(vmcb) | _event.pending_dbg);
+
         /* fall through */
     case X86_EXC_BP:
         if ( curr->domain->debugger_attached )
         {
             /* Debug/Int3: Trap to debugger. */
+            if ( _event.vector == X86_EXC_BP )
+            {
+                /* N.B. Can't use __update_guest_eip() for risk of recusion. */
+                regs->rip += _event.insn_len;
+                regs->eflags &= ~X86_EFLAGS_RF;
+                curr->arch.gdbsx_vcpu_event = X86_EXC_BP;
+            }
+
             domain_pause_for_debugger();
             return;
         }
+        else
+        {
+            int rc = hvm_monitor_debug(regs->rip,
+                                       _event.vector == X86_EXC_DB
+                                       ? HVM_MONITOR_DEBUG_EXCEPTION
+                                       : HVM_MONITOR_SOFTWARE_BREAKPOINT,
+                                       _event.type, _event.insn_len,
+                                       _event.pending_dbg);
+            if ( rc < 0 )
+            {
+                gprintk(XENLOG_ERR, "Monitor debug error %d\n", rc);
+                return svm_crash_or_fault(curr);
+            }
+            if ( rc )
+                return; /* VCPU paused.  Wait for monitor. */
+        }
         break;
 
     case X86_EXC_PF:
@@ -2730,67 +2761,46 @@ void svm_vmexit_handler(void)
 
     case VMEXIT_ICEBP:
     case VMEXIT_EXCEPTION_DB:
-        if ( !v->domain->debugger_attached )
+    case VMEXIT_EXCEPTION_BP:
+    {
+        unsigned int vec, type, len, extra;
+
+        switch ( exit_reason )
         {
-            unsigned int trap_type;
+        case VMEXIT_ICEBP:
+            vec   = X86_EXC_DB;
+            type  = X86_EVENTTYPE_PRI_SW_EXCEPTION;
+            len   = svm_get_insn_len(v, INSTR_ICEBP);
+            extra = 0;
+            break;
 
-            if ( likely(exit_reason != VMEXIT_ICEBP) )
-            {
-                trap_type = X86_EVENTTYPE_HW_EXCEPTION;
-                insn_len = 0;
-            }
-            else
-            {
-                trap_type = X86_EVENTTYPE_PRI_SW_EXCEPTION;
-                insn_len = svm_get_insn_len(v, INSTR_ICEBP);
+        case VMEXIT_EXCEPTION_DB:
+            vec   = X86_EXC_DB;
+            type  = X86_EVENTTYPE_HW_EXCEPTION;
+            len   = 0;
+            /* #DB - Hardware has already updated %dr6 for us. */
+            extra = vmcb_get_dr6(vmcb) ^ X86_DR6_DEFAULT;
+            break;
 
-                if ( !insn_len )
-                    break;
-            }
+        case VMEXIT_EXCEPTION_BP:
+            vec   = X86_EXC_BP;
+            type  = X86_EVENTTYPE_SW_EXCEPTION;
+            len   = svm_get_insn_len(v, INSTR_INT3);
+            extra = 0; /* N/A */
+            break;
 
-            rc = hvm_monitor_debug(regs->rip,
-                                   HVM_MONITOR_DEBUG_EXCEPTION,
-                                   trap_type, insn_len, 0);
-            if ( rc < 0 )
-                goto unexpected_exit_type;
-            if ( !rc )
-                hvm_inject_exception(X86_EXC_DB,
-                                     trap_type, insn_len, X86_EVENT_NO_EC,
-                                     exit_reason == VMEXIT_ICEBP ? 0 :
-                                     /* #DB - Hardware already updated dr6. */
-                                     vmcb_get_dr6(vmcb) ^ X86_DR6_DEFAULT);
+        default:
+            ASSERT_UNREACHABLE();
+            goto unexpected_exit_type;
         }
-        else
-            domain_pause_for_debugger();
-        break;
 
-    case VMEXIT_EXCEPTION_BP:
-        insn_len = svm_get_insn_len(v, INSTR_INT3);
-
-        if ( insn_len == 0 )
-             break;
+        /* svm_get_insn_len() failure.  #GP queued up. */
+        if ( type >= X86_EVENTTYPE_SW_INTERRUPT && !len )
+            break;
 
-        if ( v->domain->debugger_attached )
-        {
-            /* AMD Vol2, 15.11: INT3, INTO, BOUND intercepts do not update RIP. */
-            __update_guest_eip(regs, insn_len);
-            current->arch.gdbsx_vcpu_event = X86_EXC_BP;
-            domain_pause_for_debugger();
-        }
-        else
-        {
-           rc = hvm_monitor_debug(regs->rip,
-                                  HVM_MONITOR_SOFTWARE_BREAKPOINT,
-                                  X86_EVENTTYPE_SW_EXCEPTION,
-                                  insn_len, 0);
-           if ( rc < 0 )
-               goto unexpected_exit_type;
-           if ( !rc )
-               hvm_inject_exception(X86_EXC_BP,
-                                    X86_EVENTTYPE_SW_EXCEPTION,
-                                    insn_len, X86_EVENT_NO_EC, 0 /* N/A */);
-        }
+        hvm_inject_exception(vec, type, len, X86_EVENT_NO_EC, extra);
         break;
+    }
 
     case VMEXIT_EXCEPTION_NM:
         svm_fpu_dirty_intercept();
xen/arch/x86/hvm/vmx/vmx.c +42 -60

150 lines not on this page · raw message

Signature · 1 line
2.41.0

A message that is not in the archive (sent off-list or not yet received): CABfawh=-cTX8PfQpiT1UCXMKmB_MMy-Oc0YJUDLnU7X8=6vEeg@mail.gmail.com

author
#9 lore raw

Re: [PATCH 3/5] x86/hvm: RFC - PROBABLY BROKEN - Defer all debugging/monitor actions to {svm,vmx}_inject_event()

Quoted 14 lines · Tamas K Lengyel
Quoted 13 lines · Jinoh Kang

This is RFC because it probably breaks introspection, as injection replies from the introspection engine will (probably, but I haven't confirmed) trigger new monitor events.

First of all, monitoring emulated debug events is a change in behaviour, although IMO it is more of a bugfix than a new feature. Also, similar changes will happen to other monitored events as we try to unify the intercept/emulation paths.

As for the recursive triggering of monitor events, I was considering extending the monitor infrastructure to have a "in monitor reply" boolean which causes hvm_monitor_debug() to ignore the recursive request.

Quoted 6 lines
Quoted 2 lines

Does this plan sound ok, or have I missed something more subtle with monitor handling?

Sorry but from a monitor perspective this is a no-go. AFAIU injection will only happen once the vCPU gets rescheduled after the exit, which may take a long time, introducing potentially significant delay.

AFAIU hvm_monitor_debug() call still happens *before* VMEntry when the softirq processing and actual injection takes place. vmx_inject_event() is responsible for queueing an event to the inject slot, not processing what is already injected.

Quoted 3 lines

The monitor agent itself may need to inject events or alter what gets injected, so now it may need to undo the injection of the event from where it was called from - if that's even possible.

The monitor still gets the chance to drop the injection; 'rc > 0' from hvm_monitor_debug() still results in the injection request simply being ignored.

Quoted 6 lines

I also find the
change in behavior whereby we would be getting events for emulated events on the monitor ring equiavelly problematic, making it very hard to reason about the state of the VM on the monitor side. If that could be made optional that a monitor user can subscribe to that would be fine, but doing so unconditionally is also no-go.

We could use an additional field to 'struct x86_event' to indicate the source of the injected exception--be it VMExit, emulator, or monitor-triggered events. I'd like to hear about Andrew's opinion on this first, though.

Quoted 2 lines

Tamas

Signature · 2 lines
Sincerely,
Jinoh Kang
author
#10 lore raw

PATCH 4/5 x86: Fix merging of new status bits into %dr6

In reply to Jinoh Kang

From: Andrew Cooper <[email protected]>

The current logic used to update %dr6 when injecting #DB is buggy. The architectural behaviour is to overwrite B{0..3} (rather than accumulate) and accumulate all other bits.

Introduce a new merge_dr6() helper, which also takes care of handing RTM correctly.

---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Jun Nakajima <[email protected]>
CC: Kevin Tian <[email protected]>

---
 xen/arch/x86/hvm/svm/svm.c           |  3 ++-
 xen/arch/x86/hvm/vmx/vmx.c           |  3 ++-
 xen/arch/x86/include/asm/debugreg.h  | 30 +++++++++++++++++++++++++++-
 xen/arch/x86/include/asm/x86-defns.h | 10 ----------
 xen/arch/x86/pv/traps.c              |  3 ++-
 5 files changed, 35 insertions(+), 14 deletions(-)
xen/arch/x86/hvm/svm/svm.c +2 -1
index bd3adde0ec..7a5652f3df 100644
@@ -1338,7 +1338,8 @@ static void cf_check svm_inject_event(const struct x86_event *event)
          * Item 2 is done by hardware when injecting a #DB exception.
          */
         __restore_debug_registers(vmcb, curr);
-        vmcb_set_dr6(vmcb, vmcb_get_dr6(vmcb) | _event.pending_dbg);
+        vmcb_set_dr6(vmcb, merge_dr6(vmcb_get_dr6(vmcb), _event.pending_dbg,
+                                     curr->domain->arch.cpuid->feat.rtm));
 
         /* fall through */
     case X86_EXC_BP:
xen/arch/x86/hvm/vmx/vmx.c +2 -1
index 65166348f1..1fd902ed74 100644
@@ -2031,7 +2031,8 @@ static void cf_check vmx_inject_event(const struct x86_event *event)
          * All actions are left up to the hypervisor to perform.
          */
         __restore_debug_registers(curr);
-        write_debugreg(6, read_debugreg(6) | event->pending_dbg);
+        write_debugreg(6, merge_dr6(read_debugreg(6), event->pending_dbg,
+                                    curr->domain->arch.cpuid->feat.rtm));
 
         if ( !nestedhvm_vcpu_in_guestmode(curr) ||
              !nvmx_intercepts_exception(curr, X86_EXC_DB, _event.error_code) )
xen/arch/x86/include/asm/debugreg.h +29 -1
index a1df74c02e..fd32846397 100644
@@ -23,6 +23,14 @@
 #define X86_DR6_BT              (1u << 15)  /* Task switch             */
 #define X86_DR6_RTM             (1u << 16)  /* #DB/#BP in RTM region   */
 
+#define X86_DR6_BP_MASK                                 \
+    (X86_DR6_B0 | X86_DR6_B1 | X86_DR6_B2 | X86_DR6_B3)
+
+#define X86_DR6_KNOWN_MASK                                              \
+    (X86_DR6_BP_MASK | X86_DR6_BD | X86_DR6_BS | X86_DR6_BT | X86_DR6_RTM)
+
+#define X86_DR6_DEFAULT         0xffff0ff0  /* Default %dr6 value. */
+
 #define DR_TRAP0        (0x1)           /* db0 */
 #define DR_TRAP1        (0x2)           /* db1 */
 #define DR_TRAP2        (0x4)           /* db2 */
@@ -30,7 +38,6 @@
 #define DR_STEP         (0x4000)        /* single-step */
 #define DR_SWITCH       (0x8000)        /* task switch */
 #define DR_NOT_RTM      (0x10000)       /* clear: #BP inside RTM region */
-#define DR_STATUS_RESERVED_ONE  0xffff0ff0UL /* Reserved, read as one */
 
 /* Now define a bunch of things for manipulating the control register.
    The top two bytes of the control register consist of 4 fields of 4
@@ -74,6 +81,8 @@
 #define DR_RTM_ENABLE            (0x00000800UL) /* RTM debugging enable */
 #define DR_GENERAL_DETECT        (0x00002000UL) /* General detect enable */
 
+#define X86_DR7_DEFAULT         0x00000400  /* Default %dr7 value. */
+
 #define write_debugreg(reg, val) do {                       \
     unsigned long __val = val;                              \
     asm volatile ( "mov %0,%%db" #reg : : "r" (__val) );    \
@@ -102,6 +111,25 @@ static inline unsigned long adjust_dr6_rsvd(unsigned long dr6, bool rtm)
     return dr6;
 }
 
+static inline unsigned long merge_dr6(unsigned long dr6, unsigned long new,
+                                      bool rtm)
+{
+    /* Flip dr6 to have positive polarity. */
+    dr6 ^= X86_DR6_DEFAULT;
+
+    /* Sanity check that only known values are passed in. */
+    ASSERT(!(dr6 & ~X86_DR6_KNOWN_MASK));
+    ASSERT(!(new & ~X86_DR6_KNOWN_MASK));
+
+    /* Breakpoints 0-3 overridden.  BD, BS, BT and RTM accumulate. */
+    dr6 = (dr6 & ~X86_DR6_BP_MASK) | new;
+
+    /* Flip dr6 back to having default polarity. */
+    dr6 ^= X86_DR6_DEFAULT;
+
+    return adjust_dr6_rsvd(dr6, rtm);
+}
+
 static inline unsigned long adjust_dr7_rsvd(unsigned long dr7, bool rtm)
 {
     /*
xen/arch/x86/include/asm/x86-defns.h +0 -10
index e350227e57..7fbc74850a 100644
@@ -100,16 +100,6 @@
 #define X86_XCR0_LWP_POS          62
 #define X86_XCR0_LWP              (1ULL << X86_XCR0_LWP_POS)
 
-/*
- * Debug status flags in DR6.
- */
-#define X86_DR6_DEFAULT         0xffff0ff0  /* Default %dr6 value. */
-
-/*
- * Debug control flags in DR7.
- */
-#define X86_DR7_DEFAULT         0x00000400  /* Default %dr7 value. */
-
 /*
  * Invalidation types for the INVPCID instruction.
  */
xen/arch/x86/pv/traps.c +2 -1
index 4f5641a47c..65b41e6115 100644
@@ -66,7 +66,8 @@ void pv_inject_event(const struct x86_event *event)
         break;
 
     case X86_EXC_DB:
-        curr->arch.dr6 |= event->pending_dbg;
+        curr->arch.dr6 = merge_dr6(curr->arch.dr6, event->pending_dbg,
+                                   curr->domain->arch.cpuid->feat.rtm);
         /* Fallthrough */
 
     default:
Signature · 1 line
2.41.0
author
#11 lore raw

PATCH 5/5 x86/dbg: Cleanup of legacy dr6 constants

In reply to Jinoh Kang

From: Andrew Cooper <[email protected]>

Replace the few remaining uses with X86_DR6_* constants.

---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>

---
 xen/arch/x86/hvm/vmx/vmx.c          |  2 +-
 xen/arch/x86/include/asm/debugreg.h | 17 -----------------
 xen/arch/x86/pv/emul-priv-op.c      |  2 +-
 xen/arch/x86/traps.c                |  2 +-
 4 files changed, 3 insertions(+), 20 deletions(-)
xen/arch/x86/hvm/vmx/vmx.c +1 -1
index 1fd902ed74..43b57a444e 100644
@@ -4306,7 +4306,7 @@ void vmx_vmexit_handler(struct cpu_user_regs *regs)
 
                     __vmread(GUEST_PENDING_DBG_EXCEPTIONS, &pending_dbg);
                     __vmwrite(GUEST_PENDING_DBG_EXCEPTIONS,
-                              pending_dbg | DR_STEP);
+                              pending_dbg | X86_DR6_BS);
                 }
             }
 
xen/arch/x86/include/asm/debugreg.h +0 -17
index fd32846397..1061fdaaae 100644
@@ -1,15 +1,6 @@
 #ifndef _X86_DEBUGREG_H
 #define _X86_DEBUGREG_H
 
-
-/* Indicate the register numbers for a number of the specific
-   debug registers.  Registers 0-3 contain the addresses we wish to trap on */
-
-#define DR_FIRSTADDR 0
-#define DR_LASTADDR  3
-#define DR_STATUS    6
-#define DR_CONTROL   7
-
 /*
  * DR6 status bits.
  *   N.B. For backwards compatibility, X86_DR6_RTM has inverted polarity.
@@ -31,14 +22,6 @@
 
 #define X86_DR6_DEFAULT         0xffff0ff0  /* Default %dr6 value. */
 
-#define DR_TRAP0        (0x1)           /* db0 */
-#define DR_TRAP1        (0x2)           /* db1 */
-#define DR_TRAP2        (0x4)           /* db2 */
-#define DR_TRAP3        (0x8)           /* db3 */
-#define DR_STEP         (0x4000)        /* single-step */
-#define DR_SWITCH       (0x8000)        /* task switch */
-#define DR_NOT_RTM      (0x10000)       /* clear: #BP inside RTM region */
-
 /* Now define a bunch of things for manipulating the control register.
    The top two bytes of the control register consist of 4 fields of 4
    bits - each field corresponds to one of the four debug registers,
xen/arch/x86/pv/emul-priv-op.c +1 -1
index 72d0514e74..78a1f4aff7 100644
@@ -1359,7 +1359,7 @@ int pv_emulate_privileged_op(struct cpu_user_regs *regs)
     {
     case X86EMUL_OKAY:
         if ( ctxt.ctxt.retire.singlestep )
-            ctxt.bpmatch |= DR_STEP;
+            ctxt.bpmatch |= X86_DR6_BS;
 
         if ( ctxt.bpmatch &&
              !(curr->arch.pv.trap_bounce.flags & TBF_EXCEPTION) )
xen/arch/x86/traps.c +1 -1
index 640b60e0e5..829741ff78 100644
@@ -1955,7 +1955,7 @@ void do_debug(struct cpu_user_regs *regs)
          * If however we do, safety measures need to be enacted.  Use a big
          * hammer and clear all debug settings.
          */
-        if ( dr6 & (DR_TRAP3 | DR_TRAP2 | DR_TRAP1 | DR_TRAP0) )
+        if ( dr6 & X86_DR6_BP_MASK )
         {
             unsigned int bp, dr7 = read_debugreg(7);
 
Signature · 1 line
2.41.0
#12 lore raw
In reply to Jinoh Kang
Quoted 17 lines · Jinoh Kang

This is a rebased version of Andrew Cooper's debugging facilities patch:
https://lore.kernel.org/xen-devel/[email protected]/

Quoted 14 lines

So this started as a small fix for the vmentry failure (penultimate patch), and has snowballed...

I'm fairly confident that everything involving DEBUGCTL.BTF is broken, and there are definitely bugs with configuring DEBUGCTL.RTM (which really isn't helped by the fact that the GCC TSX intrinsics render the resulting code un-debuggable.) I'll defer fixing these swamps for now.

The first 4 patches probably want backporting to the stable trees, so I've taken care to move them ahead of patch 6 for backport reasons. While all fixes would ideally be backported, I can't find a way of fixing %dr6 merging (as it needs to be done precicely once) without a behavioural change in the monitor subsystem.

Quoted 6 lines
Quoted 2 lines

Patch 8 probably breaks introspection, so can't be taken at this point. See that patch for discussion of the problem and my best guess at a solution.

6 out of 11 patches from the 2018 patch series above, including the vmentry failure fix, have already been committed. This covers the remaining 5 patches.

One important formal question: Where did Andrew's S-o-b go on all of the patches?

Jan

author
#13 lore raw
Quoted 2 lines · Jan Beulich

One important formal question: Where did Andrew's S-o-b go on all of the patches?

Thanks for catching it. Looks like I confused the submission process with that of another (non-LF) project. I'll re-read the docs to see if I missed something else w.r.t. tagging rules.

Quoted 2 lines

Jan

Signature · 2 lines
Sincerely,
Jinoh Kang