Skip to contentSkip to conversation

xen: fix and cleanup domctl handling

PATCH In review Juergen Gross 3 participants 13 messages Apr 19, 2022

lore
Share & tools
lore
#0 lore raw

v3 cover xen: fix and cleanup domctl handling

A fix of a hypervisor crash in domctl handling and some related cleanup.

Juergen Gross (3):
xen: fix XEN_DOMCTL_gdbsx_guestmemio crash
xen: cleanup gdbsx_guest_mem_io() call xen/iommu: cleanup iommu related domctl handling

 xen/arch/arm/domctl.c               | 11 +----------
 xen/arch/x86/debug.c                | 12 +++---------
 xen/arch/x86/domctl.c               |  8 ++++----
 xen/arch/x86/include/asm/debugger.h |  2 +-
 xen/common/domctl.c                 |  8 +++++++-
 5 files changed, 16 insertions(+), 25 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH v3 1/3 xen: fix XEN_DOMCTL_gdbsx_guestmemio crash

A hypervisor built without CONFIG_GDBSX will crash in case the XEN_DOMCTL_gdbsx_guestmemio domctl is being called, as the call will end up in iommu_do_domctl() with d == NULL:

(XEN) CPU: 6
(XEN) RIP: e008:[<ffff82d040269984>] iommu_do_domctl+0x4/0x30
(XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (d0v0)
(XEN) rax: 00000000000003e8 rbx: ffff830856277ef8 rcx: ffff830856277fff
...
(XEN) Xen call trace:

(XEN)    [<ffff82d040269984>] R iommu_do_domctl+0x4/0x30
(XEN)    [<ffff82d04035cd5f>] S arch_do_domctl+0x7f/0x2330
(XEN)    [<ffff82d040239e46>] S do_domctl+0xe56/0x1930
(XEN)    [<ffff82d040238ff0>] S do_domctl+0/0x1930
(XEN)    [<ffff82d0402f8c59>] S pv_hypercall+0x99/0x110
(XEN)    [<ffff82d0402f5161>] S arch/x86/pv/domain.c#_toggle_guest_pt+0x11/0x90
(XEN)    [<ffff82d040366288>] S lstar_enter+0x128/0x130

(XEN)
(XEN) Pagetable walk from 0000000000000144:
(XEN) L4[0x000] = 0000000000000000 ffffffffffffffff (XEN)
(XEN) **************************************** (XEN) Panic on CPU 6:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0000]
(XEN) Faulting linear address: 0000000000000144

Fix this issue by making sure the domain pointer has a sane value.

Fixes e726a82ca0dc ("xen: make gdbsx support configurable")

---
V2:
- use gdbsx_guest_mem_io() interface modification (Jan Beulich) V3:
- avoid d being NULL (Andrew Cooper)

---
 xen/common/domctl.c | 1 -
 1 file changed, 1 deletion(-)
xen/common/domctl.c +0 -1
index 57135d4478..5879117580 100644
@@ -308,7 +308,6 @@ long cf_check do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
         if ( op->domain == DOMID_INVALID )
         {
     case XEN_DOMCTL_createdomain:
-    case XEN_DOMCTL_gdbsx_guestmemio:
             d = NULL;
             break;
         }
Signature · 1 line
2.34.1
Reviewed-by on v3 1/3
#2 lore raw

Re: [PATCH v3 1/3] xen: fix XEN_DOMCTL_gdbsx_guestmemio crash

Quoted 27 lines · Juergen Gross

A hypervisor built without CONFIG_GDBSX will crash in case the XEN_DOMCTL_gdbsx_guestmemio domctl is being called, as the call will end up in iommu_do_domctl() with d == NULL:

(XEN) CPU: 6
(XEN) RIP: e008:[<ffff82d040269984>] iommu_do_domctl+0x4/0x30
(XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (d0v0)
(XEN) rax: 00000000000003e8 rbx: ffff830856277ef8 rcx: ffff830856277fff
...
(XEN) Xen call trace:

(XEN)    [<ffff82d040269984>] R iommu_do_domctl+0x4/0x30
(XEN)    [<ffff82d04035cd5f>] S arch_do_domctl+0x7f/0x2330
(XEN)    [<ffff82d040239e46>] S do_domctl+0xe56/0x1930
(XEN)    [<ffff82d040238ff0>] S do_domctl+0/0x1930
(XEN)    [<ffff82d0402f8c59>] S pv_hypercall+0x99/0x110
(XEN)    [<ffff82d0402f5161>] S arch/x86/pv/domain.c#_toggle_guest_pt+0x11/0x90
(XEN)    [<ffff82d040366288>] S lstar_enter+0x128/0x130

(XEN)
(XEN) Pagetable walk from 0000000000000144:
(XEN) L4[0x000] = 0000000000000000 ffffffffffffffff (XEN)
(XEN) **************************************** (XEN) Panic on CPU 6:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0000]
(XEN) Faulting linear address: 0000000000000144

Quoted 5 lines

Fix this issue by making sure the domain pointer has a sane value.

Fixes e726a82ca0dc ("xen: make gdbsx support configurable")
#3 lore raw

Re: [PATCH v3 1/3] xen: fix XEN_DOMCTL_gdbsx_guestmemio crash

In reply to Juergen Gross
Quoted 27 lines · Juergen Gross

A hypervisor built without CONFIG_GDBSX will crash in case the XEN_DOMCTL_gdbsx_guestmemio domctl is being called, as the call will end up in iommu_do_domctl() with d == NULL:

(XEN) CPU: 6
(XEN) RIP: e008:[<ffff82d040269984>] iommu_do_domctl+0x4/0x30
(XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (d0v0)
(XEN) rax: 00000000000003e8 rbx: ffff830856277ef8 rcx: ffff830856277fff
...
(XEN) Xen call trace:

(XEN)    [<ffff82d040269984>] R iommu_do_domctl+0x4/0x30
(XEN)    [<ffff82d04035cd5f>] S arch_do_domctl+0x7f/0x2330
(XEN)    [<ffff82d040239e46>] S do_domctl+0xe56/0x1930
(XEN)    [<ffff82d040238ff0>] S do_domctl+0/0x1930
(XEN)    [<ffff82d0402f8c59>] S pv_hypercall+0x99/0x110
(XEN)    [<ffff82d0402f5161>] S arch/x86/pv/domain.c#_toggle_guest_pt+0x11/0x90
(XEN)    [<ffff82d040366288>] S lstar_enter+0x128/0x130

(XEN)
(XEN) Pagetable walk from 0000000000000144:
(XEN) L4[0x000] = 0000000000000000 ffffffffffffffff (XEN)
(XEN) **************************************** (XEN) Panic on CPU 6:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0000]
(XEN) Faulting linear address: 0000000000000144

Quoted 5 lines

Fix this issue by making sure the domain pointer has a sane value.

Fixes e726a82ca0dc ("xen: make gdbsx support configurable")

Thanks, but I was hoping for a bit of discussion on the use of DOMID_IDLE.

It used to be permitted to pass DOMID_IDLE to dbg_rw_mem() to access Xen memory, which is why the XEN_DOMCTL_gdbsx_guestmemio special case existed.

It turns out that it was also e726a82ca0dc which dropped the ability to use DOMID_IDLE, meaning that this fix is a missing hunk from the original change too.

This is relevant backport information, and would have created complexities if they hadn't been the same changeset.

So, now about:

"It used to be permitted to pass DOMID_IDLE to dbg_rw_mem(), which is why the special case excluding domid checks exists.  Now that it is only permitted to pass proper domids, remove the special case, thus making 'd' always valid."

?

Can be fixed on commit, so Reviewed-by: Andrew Cooper <[email protected]> for everything else.

~Andrew

#4 lore raw

Re: [PATCH v3 1/3] xen: fix XEN_DOMCTL_gdbsx_guestmemio crash

Quoted 46 lines · Andrew Cooper
Quoted 27 lines · Juergen Gross

A hypervisor built without CONFIG_GDBSX will crash in case the XEN_DOMCTL_gdbsx_guestmemio domctl is being called, as the call will end up in iommu_do_domctl() with d == NULL:

(XEN) CPU: 6
(XEN) RIP: e008:[<ffff82d040269984>] iommu_do_domctl+0x4/0x30
(XEN) RFLAGS: 0000000000010202 CONTEXT: hypervisor (d0v0)
(XEN) rax: 00000000000003e8 rbx: ffff830856277ef8 rcx: ffff830856277fff
...
(XEN) Xen call trace:

(XEN)    [<ffff82d040269984>] R iommu_do_domctl+0x4/0x30
(XEN)    [<ffff82d04035cd5f>] S arch_do_domctl+0x7f/0x2330
(XEN)    [<ffff82d040239e46>] S do_domctl+0xe56/0x1930
(XEN)    [<ffff82d040238ff0>] S do_domctl+0/0x1930
(XEN)    [<ffff82d0402f8c59>] S pv_hypercall+0x99/0x110
(XEN)    [<ffff82d0402f5161>] S arch/x86/pv/domain.c#_toggle_guest_pt+0x11/0x90
(XEN)    [<ffff82d040366288>] S lstar_enter+0x128/0x130

(XEN)
(XEN) Pagetable walk from 0000000000000144:
(XEN) L4[0x000] = 0000000000000000 ffffffffffffffff (XEN)
(XEN) **************************************** (XEN) Panic on CPU 6:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0000]
(XEN) Faulting linear address: 0000000000000144

Quoted 5 lines

Fix this issue by making sure the domain pointer has a sane value.

Fixes e726a82ca0dc ("xen: make gdbsx support configurable")

Thanks, but I was hoping for a bit of discussion on the use of DOMID_IDLE.

It used to be permitted to pass DOMID_IDLE to dbg_rw_mem() to access Xen memory, which is why the XEN_DOMCTL_gdbsx_guestmemio special case existed.

It turns out that it was also e726a82ca0dc which dropped the ability to use DOMID_IDLE, meaning that this fix is a missing hunk from the original change too.

This is relevant backport information, and would have created complexities if they hadn't been the same changeset.

Quoted 8 lines

So, now about:

"It used to be permitted to pass DOMID_IDLE to dbg_rw_mem(), which is why the special case excluding domid checks exists.  Now that it is only permitted to pass proper domids, remove the special case, thus making 'd' always valid."

?

Fine with me.

Quoted 3 lines

Can be fixed on commit, so Reviewed-by: Andrew Cooper <[email protected]> for everything else.

Thanks,

Juergen

#5 lore raw

PATCH v3 2/3 xen: cleanup gdbsx_guest_mem_io() call

In reply to Juergen Gross

Modify the gdbsx_guest_mem_io() interface to take the already known domain pointer as parameter instead of the domid. This enables to remove some more code further down the call tree.

---
V2:
- split off from previous patch (Andrew Cooper)

---
 xen/arch/x86/debug.c                | 12 +++---------
 xen/arch/x86/domctl.c               |  6 +++---
 xen/arch/x86/include/asm/debugger.h |  2 +-
 3 files changed, 7 insertions(+), 13 deletions(-)
xen/arch/x86/debug.c +3 -9
index d90dc93056..62fbabb084 100644
@@ -159,17 +159,11 @@ static unsigned int dbg_rw_guest_mem(struct domain *dp, unsigned long addr,
  * Returns: number of bytes remaining to be copied.
  */
 unsigned int dbg_rw_mem(unsigned long gva, XEN_GUEST_HANDLE_PARAM(void) buf,
-                        unsigned int len, domid_t domid, bool toaddr,
+                        unsigned int len, struct domain *d, bool toaddr,
                         uint64_t pgd3)
 {
-    struct domain *d = rcu_lock_domain_by_id(domid);
-
-    if ( d )
-    {
-        if ( !d->is_dying )
-            len = dbg_rw_guest_mem(d, gva, buf, len, toaddr, pgd3);
-        rcu_unlock_domain(d);
-    }
+    if ( d && !d->is_dying )
+        len = dbg_rw_guest_mem(d, gva, buf, len, toaddr, pgd3);
 
     return len;
 }
xen/arch/x86/domctl.c +3 -3
index e49f9e91b9..a6aae500a3 100644
@@ -38,10 +38,10 @@
 #include <asm/cpuid.h>
 
 #ifdef CONFIG_GDBSX
-static int gdbsx_guest_mem_io(domid_t domid, struct xen_domctl_gdbsx_memio *iop)
+static int gdbsx_guest_mem_io(struct domain *d, struct xen_domctl_gdbsx_memio *iop)
 {
     iop->remain = dbg_rw_mem(iop->gva, guest_handle_from_ptr(iop->uva, void),
-                             iop->len, domid, iop->gwr, iop->pgd3val);
+                             iop->len, d, iop->gwr, iop->pgd3val);
 
     return iop->remain ? -EFAULT : 0;
 }
@@ -828,7 +828,7 @@ long arch_do_domctl(
 #ifdef CONFIG_GDBSX
     case XEN_DOMCTL_gdbsx_guestmemio:
         domctl->u.gdbsx_guest_memio.remain = domctl->u.gdbsx_guest_memio.len;
-        ret = gdbsx_guest_mem_io(domctl->domain, &domctl->u.gdbsx_guest_memio);
+        ret = gdbsx_guest_mem_io(d, &domctl->u.gdbsx_guest_memio);
         if ( !ret )
            copyback = true;
         break;
xen/arch/x86/include/asm/debugger.h +1 -1
index 99803bfd0c..221bcde137 100644
@@ -94,7 +94,7 @@ static inline bool debugger_trap_entry(
 
 #ifdef CONFIG_GDBSX
 unsigned int dbg_rw_mem(unsigned long gva, XEN_GUEST_HANDLE_PARAM(void) buf,
-                        unsigned int len, domid_t domid, bool toaddr,
+                        unsigned int len, struct domain *d, bool toaddr,
                         uint64_t pgd3);
 #endif
 
Signature · 1 line
2.34.1
Reviewed-by on v3 2/3
#7 lore raw

Re: [PATCH v3 2/3] xen: cleanup gdbsx_guest_mem_io() call

In reply to Juergen Gross
Quoted 6 lines · Juergen Gross

Modify the gdbsx_guest_mem_io() interface to take the already known domain pointer as parameter instead of the domid. This enables to remove some more code further down the call tree.

I'll rebase the other debugger cleanup over this.

#8 lore raw

PATCH v3 3/3 xen/iommu: cleanup iommu related domctl handling

In reply to Juergen Gross

Today iommu_do_domctl() is being called from arch_do_domctl() in the "default:" case of a switch statement. This has led already to crashes due to unvalidated parameters.

Fix that by moving the call of iommu_do_domctl() to the main switch statement of do_domctl().

---
Another possibility would even be to merge iommu_do_domctl() completely into do_domctl(), but I wanted to start with a less intrusive variant. V3:
- new patch

---
 xen/arch/arm/domctl.c | 11 +----------
 xen/arch/x86/domctl.c |  2 +-
 xen/common/domctl.c   |  7 +++++++
 3 files changed, 9 insertions(+), 11 deletions(-)
xen/arch/arm/domctl.c +1 -10
index 6245af6d0b..1baf25c3d9 100644
@@ -176,16 +176,7 @@ long arch_do_domctl(struct xen_domctl *domctl, struct domain *d,
         return rc;
     }
     default:
-    {
-        int rc;
-
-        rc = subarch_do_domctl(domctl, d, u_domctl);
-
-        if ( rc == -ENOSYS )
-            rc = iommu_do_domctl(domctl, d, u_domctl);
-
-        return rc;
-    }
+        return subarch_do_domctl(domctl, d, u_domctl);
     }
 }
 
xen/arch/x86/domctl.c +1 -1
index a6aae500a3..c9699bb868 100644
@@ -1380,7 +1380,7 @@ long arch_do_domctl(
         break;
 
     default:
-        ret = iommu_do_domctl(domctl, d, u_domctl);
+        ret = -ENOSYS;
         break;
     }
 
xen/common/domctl.c +7 -0
index 5879117580..0a866e3132 100644
@@ -871,6 +871,13 @@ long cf_check do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
             copyback = 1;
         break;
 
+    case XEN_DOMCTL_assign_device:
+    case XEN_DOMCTL_test_assign_device:
+    case XEN_DOMCTL_deassign_device:
+    case XEN_DOMCTL_get_device_group:
+        ret = iommu_do_domctl(op, d, u_domctl);
+        break;
+
     default:
         ret = arch_do_domctl(op, d, u_domctl);
         break;
Signature · 1 line
2.34.1
#9 lore raw

Re: [PATCH v3 3/3] xen/iommu: cleanup iommu related domctl handling

xen/common/domctl.c long cf_check do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl) · line 877

Quoted 10 lines · Juergen Gross
xen/common/domctl.c +4 -0
@@ -871,6 +871,13 @@ long cf_check do_domctl(XEN_GUEST_HANDLE_PARAM(xen_domctl_t) u_domctl)
             copyback = 1;
         break;
 
+    case XEN_DOMCTL_assign_device:
+    case XEN_DOMCTL_test_assign_device:
+    case XEN_DOMCTL_deassign_device:
+    case XEN_DOMCTL_get_device_group:

As said in reply to Andrew, I'm not convinced having these enumerated here is a good idea. But in any event the whole addition wants framing by #ifdef CONFIG_HAS_PASSTHROUGH now.

For x86 I wonder whether the adjustment wouldn't allow to drop domctl.c's including of xen/iommu.h (but perhaps it's included indirectly anyway).

Jan

Quoted 6 lines
xen/common/domctl.c +3 -0
+        ret = iommu_do_domctl(op, d, u_domctl);
+        break;
+
     default:
         ret = arch_do_domctl(op, d, u_domctl);
         break;
#10 lore raw

Re: [PATCH v3 3/3] xen/iommu: cleanup iommu related domctl handling

In reply to Juergen Gross
Quoted 7 lines · Juergen Gross

Today iommu_do_domctl() is being called from arch_do_domctl() in the "default:" case of a switch statement. This has led already to crashes due to unvalidated parameters.

Fix that by moving the call of iommu_do_domctl() to the main switch statement of do_domctl().

Quoted 6 lines

---
Another possibility would even be to merge iommu_do_domctl() completely into do_domctl(), but I wanted to start with a less intrusive variant. V3:
- new patch

I definitely prefer this approach, thanks.  In addition to being clearer, it's also faster because there isn't a long line of "do you understand this subop?" calls when we know exactly what it is.

However, I think we need stub for the !HAS_PASSTHROUGH case now that it is being called from common code.

I'd forgotten that it was used on ARM now, and yes - it absolutely should be called from somewhere common, not from the arch hooks.

~Andrew

#11 lore raw

Re: [PATCH v3 3/3] xen/iommu: cleanup iommu related domctl handling

Quoted 15 lines · Andrew Cooper
Quoted 7 lines · Juergen Gross

Today iommu_do_domctl() is being called from arch_do_domctl() in the "default:" case of a switch statement. This has led already to crashes due to unvalidated parameters.

Fix that by moving the call of iommu_do_domctl() to the main switch statement of do_domctl().

Quoted 6 lines

---
Another possibility would even be to merge iommu_do_domctl() completely into do_domctl(), but I wanted to start with a less intrusive variant. V3:
- new patch

Quoted 6 lines

I definitely prefer this approach, thanks.  In addition to being clearer, it's also faster because there isn't a long line of "do you understand this subop?" calls when we know exactly what it is.

However, I think we need stub for the !HAS_PASSTHROUGH case now that it is being called from common code.

Okay, I'll add it. Jan, are you fine with a stub?

Juergen

#12 lore raw

Re: [PATCH v3 3/3] xen/iommu: cleanup iommu related domctl handling

Quoted 16 lines · Juergen Gross
Quoted 15 lines · Andrew Cooper
Quoted 13 lines · Juergen Gross

Today iommu_do_domctl() is being called from arch_do_domctl() in the "default:" case of a switch statement. This has led already to crashes due to unvalidated parameters.

Fix that by moving the call of iommu_do_domctl() to the main switch statement of do_domctl().

---
Another possibility would even be to merge iommu_do_domctl() completely into do_domctl(), but I wanted to start with a less intrusive variant. V3:
- new patch

Quoted 8 lines
Quoted 6 lines

I definitely prefer this approach, thanks.  In addition to being clearer, it's also faster because there isn't a long line of "do you understand this subop?" calls when we know exactly what it is.

However, I think we need stub for the !HAS_PASSTHROUGH case now that it is being called from common code.

Okay, I'll add it. Jan, are you fine with a stub?

Sure.

Jan