Skip to contentSkip to conversation

- add symmetry to para and fully virtualized domain shutdown/reboot

PATCH In review Ben Thomas 3 participants 7 messages Apr 3, 2006 – Apr 4, 2006

lore
Share & tools
lore

Ben Thomas

author
#0 lore raw

PATCH 1/1 - add symmetry to para and fully virtualized domain shutdown/reboot

Add code to make handling domain poweroff/reboot symmetrical between paravirtualized and fully virtualized.

A paravirtualized domain uses sched_op to shut down and set the reason code. This will send a VIRQ_DOM_EXC, which can be handled in dom0 by control software. In some ways, this resembles SIGCHILD/waitpid, and is a reasonable model.

The fully virtualized case has qemu invoke xm directly. This is a different path than paravirtualized. It also removes decision and policy making choices from the rest of the control software and places it within qemu. When any dom0 logic eventually gets a VIRQ_DOM_EXC, the information about the domain is gone having been destroyed by xm.

It would be useful if all shutdown/reboot operations were symmetrical from domain 0's point of view. One approach would be to redefine sched_op to handle other domains than the current domain, but this seemed excessive. Another approach, which is what this patch implements, adds a DOM0 hypervisor call very similar to the existing DOM0_DESTROYDOMAIN, called DOM0_SHUTDOWNDOMAIN, which handles the shutdown and reason -- basically, just like a paravirtualized system.

Like DOM0_DESTROYDOMAIN, add a xenctrl wrapper, and have qemu call it.

With this change, both domain types shutdown/reboot using the same dom0 management logic. All control decisions are removed from qemu, which now simply handles the request and reason. At a very high level, the flow is "notify of shutdown and reason", "send VIRQ_DOM_EXC", process, and destroy domain. Paravirtualized and fully-virtualized domains would differ slightly in the first step of notification. Paravirtualized continues to use sched_op, fully virtualized uses DOM0_SHUTDOWNDOMAIN. The rest of the steps would be identical in both cases.

Or, back to the opening sentence, make the operations as symmetrical as possible.

(As a freebie, #if 0 some very verbose logging code in qemu.
Totally unrelated, but as long as I was there...)

Signature · 5 lines
------------------------------------------------------------------------
Ben Thomas                                         Virtual Iron Software
[email protected]                            Tower 1, Floor 2
978-849-1214                                       900 Chelmsford Street
                                                   Lowell, MA 01851
tools/ioemu/target-i386-dm/helper2.c +28 -9
diff -r 4ad317429111 tools/ioemu/target-i386-dm/helper2.c
@@ -409,12 +409,20 @@ void
 void
 destroy_hvm_domain(void)
 {
-    extern FILE* logfile;
-    char destroy_cmd[32];
-
-    sprintf(destroy_cmd, "xm destroy %d", domid);
-    if (system(destroy_cmd) == -1)
-        fprintf(logfile, "%s failed.!\n", destroy_cmd);
+  int	xcHandle;
+  int	sts;
+
+  xcHandle = xc_interface_open();
+  if (xcHandle < 0)
+    fprintf(logfile, "Cannot acquire xenctrl handle\n");
+  else {
+    sts = xc_domain_shutdown(xcHandle, domid, SHUTDOWN_poweroff);
+    if (sts != 0)
+      fprintf(logfile, "? xc_domain_shutdown failed to issue poweroff, sts %d\n", sts);
+    else
+      fprintf(logfile, "Issued domain %d poweroff\n", domid);
+    xc_interface_close(xcHandle);
+  }
 }
 
 fd_set wakeup_rfds;
@@ -480,13 +488,24 @@ int main_loop(void)
 
 static void qemu_hvm_reset(void *unused)
 {
-    char cmd[64];
+  int	xcHandle;
+  int	sts;
 
     /* pause domain first, to avoid repeated reboot request*/
     xc_domain_pause(xc_handle, domid);
 
-    sprintf(cmd, "xm shutdown -R %d", domid);
-    system(cmd);
+  xcHandle = xc_interface_open();
+  if (xcHandle < 0)
+    fprintf(logfile, "Cannot acquire xenctrl handle\n");
+  else {
+    sts = xc_domain_shutdown(xcHandle, domid, SHUTDOWN_reboot);
+    if (sts != 0)
+      fprintf(logfile, "? xc_domain_shutdown failed to issue reboot, sts %d\n", sts);
+    else
+      fprintf(logfile, "Issued domain %d reboot\n", domid);
+    xc_interface_close(xcHandle);
+  }
+
 }
 
 CPUState * cpu_init()
tools/ioemu/vl.c +2 -0
diff -r 4ad317429111 tools/ioemu/vl.c
@@ -2556,8 +2556,10 @@ static int set_mm_mapping(int xc_handle,
         return -1;
     }
 
+#if 0  /* Enable for debugging -- generates many lines in log file */
     for (i = 0; i < nr_pages; i++)
         fprintf(stderr, "set_map result i %x result %lx\n", i, extent_start[i]);
+#endif
 
     return 0;
 }
tools/libxc/xc_domain.c +11 -0
diff -r 4ad317429111 tools/libxc/xc_domain.c
@@ -55,6 +55,17 @@ int xc_domain_destroy(int xc_handle,
     DECLARE_DOM0_OP;
     op.cmd = DOM0_DESTROYDOMAIN;
     op.u.destroydomain.domain = (domid_t)domid;
+    return do_dom0_op(xc_handle, &op);
+}
+
+int xc_domain_shutdown(int xc_handle,
+                       uint32_t domid,
+		       int reason)
+{
+    DECLARE_DOM0_OP;
+    op.cmd = DOM0_SHUTDOWNDOMAIN;
+    op.u.shutdowndomain.domain = (domid_t)domid;
+    op.u.shutdowndomain.shutdown_reason = reason;
     return do_dom0_op(xc_handle, &op);
 }
 
tools/libxc/xenctrl.h +15 -0
diff -r 4ad317429111 tools/libxc/xenctrl.h
@@ -205,6 +205,21 @@ int xc_domain_unpause(int xc_handle,
  */
 int xc_domain_destroy(int xc_handle, 
                       uint32_t domid);
+		      
+/**
+ * This function will shutdown a domain. This is intended for use in
+ * fully-virtualized domains where this operation is analogous to the
+ * sched_op operations in a paravirtualized domain. The caller is
+ * expected to give the reason for the shutdown.
+ *
+ * @parm xc_handle a handle to an open hypervisor interface
+ * @parm domid the domain id to destroy
+ * @parm reason is the reason (SHUTDOWN_xxx) for the shutdown
+ * @return 0 on success, -1 on failure
+ */
+int xc_domain_shutdown(int xc_handle, 
+                       uint32_t domid,
+       		       int reason);
 
 int xc_vcpu_setaffinity(int xc_handle,
                         uint32_t domid,
xen/common/dom0_ops.c +17 -0
diff -r 4ad317429111 xen/common/dom0_ops.c
@@ -682,6 +682,23 @@ long do_dom0_op(GUEST_HANDLE(dom0_op_t) 
     break;
 #endif
 
+    case DOM0_SHUTDOWNDOMAIN:
+    {
+        struct domain *d = find_domain_by_id(op->u.shutdowndomain.domain);
+        ret = -ESRCH;
+        if ( d != NULL )
+        {
+            ret = -EINVAL;
+            if ( d != current->domain )
+            {
+                domain_shutdown(d, op->u.shutdowndomain.shutdown_reason);
+                ret = 0;
+            }
+	    put_domain(d);
+        }
+    }
+    break;
+
     default:
         ret = arch_do_dom0_op(op, u_dom0_op);
         break;
xen/include/public/dom0_ops.h +10 -0
diff -r 4ad317429111 xen/include/public/dom0_ops.h
@@ -470,6 +470,15 @@ typedef struct dom0_hypercall_init {
     unsigned long mfn;        /* machine frame to be initialised */
 } dom0_hypercall_init_t;
 DEFINE_GUEST_HANDLE(dom0_hypercall_init_t);
+
+#define DOM0_SHUTDOWNDOMAIN     49
+typedef struct dom0_shutdowndomain {
+    /* IN variables. */
+    domid_t domain;           /* domain to be affected */
+    int     shutdown_reason;  /* SHUTDOWN_xxx reason code */
+} dom0_shutdowndomain_t;
+
+DEFINE_GUEST_HANDLE(dom0_shutdowndomain_t);
 
 typedef struct dom0_op {
     uint32_t cmd;
@@ -512,6 +521,7 @@ typedef struct dom0_op {
         struct dom0_irq_permission    irq_permission;
         struct dom0_iomem_permission  iomem_permission;
         struct dom0_hypercall_init    hypercall_init;
+	struct dom0_shutdowndomain    shutdowndomain;
         uint8_t                       pad[128];
     } u;
 } dom0_op_t;

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#1 lore raw
Quoted 4 lines · Ben Thomas

It would be useful if all shutdown/reboot operations were symmetrical from domain 0's point of view. One approach would be to redefine sched_op to handle other domains than the current domain, but this seemed excessive.

I'd prefer an extra sched_op (maybe SCHEDOP_remote_shutdown with accompanying sched_remote_shutdown structure). This will need to be invokable by the emulator mini domain at some point in the future, so adding this as a dom0_op isn't really for the best.

-- Keir

author
#2 lore raw
Quoted 9 lines · Keir Fraser
Quoted 4 lines · Ben Thomas

It would be useful if all shutdown/reboot operations were symmetrical from domain 0's point of view. One approach would be to redefine sched_op to handle other domains than the current domain, but this seemed excessive.

Quoted 4 lines

I'd prefer an extra sched_op (maybe SCHEDOP_remote_shutdown with accompanying sched_remote_shutdown structure). This will need to be invokable by the emulator mini domain at some point in the future, so adding this as a dom0_op isn't really for the best.

-- Keir

Ok, that's workable. Is there anything else that needs modification ?

Do you foresee a need for any sort of need for filtering and/or permissioning of allowable domains as specified in remote_shutdown ? As long as a non-privileged domain may execute this op, what level of protection(s) need to exist? (That also was part of my factoring it into a DOM0 op).

I'm willing to go with anything that gets the job done. This is one of several points of asymmetry that I'd like to see get resolved.

Thanks,
-b

Signature · 5 lines
------------------------------------------------------------------------
Ben Thomas                                         Virtual Iron Software
[email protected]                            Tower 1, Floor 2
978-849-1214                                       900 Chelmsford Street
                                                   Lowell, MA 01851
#3 lore raw
Quoted 11 lines · Ben Thomas

Ok, that's workable. Is there anything else that needs modification ?

Do you foresee a need for any sort of need for filtering and/or permissioning of allowable domains as specified in remote_shutdown ? As long as a non-privileged domain may execute this op, what level of protection(s) need to exist? (That also was part of my factoring it into a DOM0 op).

I'm willing to go with anything that gets the job done. This is one of several points of asymmetry that I'd like to see get resolved.

For now, IS_PRIV() is the correct check as for a dom0_op, but it can be revisited in future.

Apart from that the patch looked okay to me.

-- Keir

#6 lore raw
In reply to Ben Thomas

thomas,

your patch is reasonable.

as my understanding, destroy is totally different from shutdown. destroy just issues a hyper-call(DOM0_OP), while shutdown writes xenstore and waits the guest handling almost everything( use sched_op hypercall for itself).

so, qemu's destroy command(kill hvm dom immediately when hang) should use the original destroy hyper-call to take place of "xm destroy".

Quoted 60 lines · Ben Thomas

Add code to make handling domain poweroff/reboot symmetrical between paravirtualized and fully virtualized.

A paravirtualized domain uses sched_op to shut down and set the reason code. This will send a VIRQ_DOM_EXC, which can be handled in dom0 by control software. In some ways, this resembles SIGCHILD/waitpid, and is a reasonable model.

The fully virtualized case has qemu invoke xm directly. This is a different path than paravirtualized. It also removes decision and policy making choices from the rest of the control software and places it within qemu. When any dom0 logic eventually gets a VIRQ_DOM_EXC, the information about the domain is gone having been destroyed by xm.

It would be useful if all shutdown/reboot operations were symmetrical from domain 0's point of view. One approach would be to redefine sched_op to handle other domains than the current domain, but this seemed excessive. Another approach, which is what this patch implements, adds a DOM0 hypervisor call very similar to the existing DOM0_DESTROYDOMAIN, called DOM0_SHUTDOWNDOMAIN, which handles the shutdown and reason -- basically, just like a paravirtualized system.

Like DOM0_DESTROYDOMAIN, add a xenctrl wrapper, and have qemu call it.

With this change, both domain types shutdown/reboot using the same dom0 management logic. All control decisions are removed from qemu, which now simply handles the request and reason. At a very high level, the flow is "notify of shutdown and reason", "send VIRQ_DOM_EXC", process, and destroy domain. Paravirtualized and fully-virtualized domains would differ slightly in the first step of notification. Paravirtualized continues to use sched_op, fully virtualized uses DOM0_SHUTDOWNDOMAIN. The rest of the steps would be identical in both cases.

Or, back to the opening sentence, make the operations as symmetrical as possible.

(As a freebie, #if 0 some very verbose logging code in qemu.
Totally unrelated, but as long as I was there...)

Signature · 11 lines
------------------------------------------------------------------------
Ben Thomas                                         Virtual Iron Software
[email protected]                            Tower 1, Floor 2
978-849-1214                                       900 Chelmsford Street
                                                   Lowell, MA 01851


_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Signature · 2 lines
thanks,
edwin
#4 lore raw
In reply to Keir Fraser
Quoted 2 lines · Keir Fraser

I'd prefer an extra sched_op (maybe SCHEDOP_remote_shutdown with

can we use common code to avoid extra same function?

Quoted 11 lines

accompanying sched_remote_shutdown structure). This will need to be invokable by the emulator mini domain at some point in the future, so adding this as a dom0_op isn't really for the best.

-- Keir

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

Signature · 2 lines
thanks,
edwin
#5 lore raw
Quoted 2 lines · Edwin Zhai
Quoted 1 line

I'd prefer an extra sched_op (maybe SCHEDOP_remote_shutdown with

can we use common code to avoid extra same function?

We need a separate interface function as it takes an extra parameter. There'll be code sharing inside Xen, of course.

-- Keir