Skip to contentSkip to conversation

qemu-ga: Introduce guest-hibernate command

RFC In review Luiz Capitulino 6 participants 12 messages Dec 8, 2011 – Dec 12, 2011

lore
Share & tools
lore
#0 lore raw

RFC 1/1 qemu-ga: Introduce guest-hibernate command

This is basically suspend to disk on a Linux guest.

---

This is an RFC because I did it as simple as possible and I'm open to suggestions...

Now, while testing this or even "echo disk > /sys/power/state" I get several funny results. Some times qemu just dies after printing that message:

"Guest moved used index from 20151 to 1"

Some times it doesn't die, but I'm unable to log into the guest: I type username & password but the terminal kind of locks (the shell doesn't run).

Some times it works...

 qapi-schema-guest.json     |   11 +++++++++++
 qga/guest-agent-commands.c |   19 +++++++++++++++++++
 2 files changed, 30 insertions(+), 0 deletions(-)
qapi-schema-guest.json +11 -0
index fde5971..2c5bbcf 100644
@@ -215,3 +215,14 @@
 ##
 { 'command': 'guest-fsfreeze-thaw',
   'returns': 'int' }
+
+##
+# @guest-hibernate
+#
+# Save RAM contents to disk and powerdown the guest.
+#
+# Notes: This command doesn't return on success.
+#
+# Since: 1.1
+##
+{ 'command': 'guest-hibernate' }
qga/guest-agent-commands.c +19 -0
index 6da9904..9dd4060 100644
@@ -550,6 +550,25 @@ int64_t qmp_guest_fsfreeze_thaw(Error **err)
 }
 #endif
 
+#define LINUX_SYS_STATE_FILE "/sys/power/state"
+
+void qmp_guest_hibernate(Error **err)
+{
+    int fd;
+
+    fd = open(LINUX_SYS_STATE_FILE, O_WRONLY);
+    if (fd < 0) {
+        error_set(err, QERR_OPEN_FILE_FAILED, LINUX_SYS_STATE_FILE);
+        return;
+    }
+
+    if (write(fd, "disk", 4) < 0) {
+        error_set(err, QERR_UNDEFINED_ERROR);
+    }
+
+    close(fd);
+}
+
 /* register init/cleanup routines for stateful command groups */
 void ga_command_state_init(GAState *s, GACommandState *cs)
 {
Signature · 1 line
1.7.8.110.g4cb5d1.dirty
#1 lore raw
Quoted 4 lines · Luiz Capitulino

This is basically suspend to disk on a Linux guest.

Now, while testing this or even "echo disk > /sys/power/state" I get several funny results. Some times qemu just dies after printing that message:

I think you should really be invoking 'pm-hibernate', because IIUC there are others tasks that need to be done, besides just telling the kernel to hibernate via sysfs.

Regards,
Daniel

Signature · 4 lines
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|
#2 lore raw
Quoted 9 lines · On Thu, 8 Dec 2011 19:07:00 +0000 "Daniel P. Berrange" <[email protected]>…
Quoted 4 lines · Luiz Capitulino

This is basically suspend to disk on a Linux guest.

Now, while testing this or even "echo disk > /sys/power/state" I get several funny results. Some times qemu just dies after printing that message:

I think you should really be invoking 'pm-hibernate', because IIUC there are others tasks that need to be done, besides just telling the kernel to hibernate via sysfs.

The good thing about it is that it will work among all distros... Maybe we should try to run '/usr/sbin/pm-hibernate' first, and only write to the sysfs file if that fails.

#3 lore raw
In reply to Luiz Capitulino

Am 08.12.2011 19:52, schrieb Luiz Capitulino:

Quoted 5 lines

This is basically suspend to disk on a Linux guest.

---

Quoted 7 lines

This is an RFC because I did it as simple as possible and I'm open to suggestions...

Now, while testing this or even "echo disk > /sys/power/state" I get several funny results. Some times qemu just dies after printing that message:

"Guest moved used index from 20151 to 1"

On s390x that happened when virtio memory was not zeroed on init...

Andreas

Signature · 2 lines
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
#4 lore raw
In reply to Luiz Capitulino
Quoted 1 line · Luiz Capitulino

This is basically suspend to disk on a Linux guest.

Nice!

Quoted 7 lines

---

This is an RFC because I did it as simple as possible and I'm open to suggestions...

Quoted 7 lines

Now, while testing this or even "echo disk> /sys/power/state" I get several funny results. Some times qemu just dies after printing that message:

"Guest moved used index from 20151 to 1"

Some times it doesn't die, but I'm unable to log into the guest: I type username& password but the terminal kind of locks (the shell doesn't run).

Triggered this pretty easily on FC15 amd64:

qemu-system-x86_64: Guest moved used index from 13024 to 80

I'll see if I can get a trace of what's happening on my end.

Quoted 39 lines

Some times it works...

  qapi-schema-guest.json     |   11 +++++++++++
  qga/guest-agent-commands.c |   19 +++++++++++++++++++
  2 files changed, 30 insertions(+), 0 deletions(-)
qapi-schema-guest.json +11 -0
index fde5971..2c5bbcf 100644
@@ -215,3 +215,14 @@
  ##
  { 'command': 'guest-fsfreeze-thaw',
    'returns': 'int' }
+
+##
+# @guest-hibernate
+#
+# Save RAM contents to disk and powerdown the guest.
+#
+# Notes: This command doesn't return on success.
+#
+# Since: 1.1
+##
+{ 'command': 'guest-hibernate' }
qga/guest-agent-commands.c +5 -0
index 6da9904..9dd4060 100644
@@ -550,6 +550,25 @@ int64_t qmp_guest_fsfreeze_thaw(Error **err)
  }
  #endif
 
+#define LINUX_SYS_STATE_FILE "/sys/power/state"
+
+void qmp_guest_hibernate(Error **err)
+{
+    int fd;

qga/guest-agent-commands.c int64_t qmp_guest_fsfreeze_thaw(Error **err) · line 567

Quoted 10 lines
qga/guest-agent-commands.c +10 -0
+
+    fd = open(LINUX_SYS_STATE_FILE, O_WRONLY);
+    if (fd<  0) {
+        error_set(err, QERR_OPEN_FILE_FAILED, LINUX_SYS_STATE_FILE);
+        return;
+    }
+
+    if (write(fd, "disk", 4)<  0) {
+        error_set(err, QERR_UNDEFINED_ERROR);
+    }

There's basically no chance of this returning, so we should issue it asynchronously like we do with qmp_guest_shutdown(). Also agreed on Daniel's point that we should use pm-hibernate, but failing back to this should be okay...assuming the virtio issue isn't indicative of some fundamental problem with that approach...

Quoted 7 lines
qga/guest-agent-commands.c +4 -0
+
+    close(fd);
+}
+
  /* register init/cleanup routines for stateful command groups */
  void ga_command_state_init(GAState *s, GACommandState *cs)
  {
#5 lore raw
Quoted 73 lines · On Thu, 08 Dec 2011 19:14:43 -0600 Michael Roth <[email protected]>…
Quoted 1 line · Luiz Capitulino

This is basically suspend to disk on a Linux guest.

Nice!

Quoted 7 lines

---

This is an RFC because I did it as simple as possible and I'm open to suggestions...

Quoted 7 lines

Now, while testing this or even "echo disk> /sys/power/state" I get several funny results. Some times qemu just dies after printing that message:

"Guest moved used index from 20151 to 1"

Some times it doesn't die, but I'm unable to log into the guest: I type username& password but the terminal kind of locks (the shell doesn't run).

Triggered this pretty easily on FC15 amd64:

qemu-system-x86_64: Guest moved used index from 13024 to 80

I'll see if I can get a trace of what's happening on my end.

Quoted 47 lines

Some times it works...

  qapi-schema-guest.json     |   11 +++++++++++
  qga/guest-agent-commands.c |   19 +++++++++++++++++++
  2 files changed, 30 insertions(+), 0 deletions(-)
qapi-schema-guest.json +11 -0
index fde5971..2c5bbcf 100644
@@ -215,3 +215,14 @@
  ##
  { 'command': 'guest-fsfreeze-thaw',
    'returns': 'int' }
+
+##
+# @guest-hibernate
+#
+# Save RAM contents to disk and powerdown the guest.
+#
+# Notes: This command doesn't return on success.
+#
+# Since: 1.1
+##
+{ 'command': 'guest-hibernate' }
qga/guest-agent-commands.c +13 -0
index 6da9904..9dd4060 100644
@@ -550,6 +550,25 @@ int64_t qmp_guest_fsfreeze_thaw(Error **err)
  }
  #endif
 
+#define LINUX_SYS_STATE_FILE "/sys/power/state"
+
+void qmp_guest_hibernate(Error **err)
+{
+    int fd;
+
+    fd = open(LINUX_SYS_STATE_FILE, O_WRONLY);
+    if (fd<  0) {
+        error_set(err, QERR_OPEN_FILE_FAILED, LINUX_SYS_STATE_FILE);
+        return;
+    }
+
+    if (write(fd, "disk", 4)<  0) {
Quoted 8 lines

qga/guest-agent-commands.c int64_t qmp_guest_fsfreeze_thaw(Error **err) · line 567

Quoted 2 lines
qga/guest-agent-commands.c +2 -0
+        error_set(err, QERR_UNDEFINED_ERROR);
+    }

There's basically no chance of this returning, so we should issue it asynchronously like we do with qmp_guest_shutdown(). Also agreed on Daniel's point that we should use pm-hibernate, but failing back to this should be okay...assuming the virtio issue isn't indicative of some fundamental problem with that approach...

Ok, I'll make these changes and submit v2.

Quoted 9 lines
Quoted 8 lines
qga/guest-agent-commands.c +4 -0
+
+    close(fd);
+}
+
  /* register init/cleanup routines for stateful command groups */
  void ga_command_state_init(GAState *s, GACommandState *cs)
  {
#6 lore raw
In reply to Luiz Capitulino
Quoted 11 lines · Luiz Capitulino

This is basically suspend to disk on a Linux guest.

---

This is an RFC because I did it as simple as possible and I'm open to suggestions...

Now, while testing this or even "echo disk> /sys/power/state" I get several funny results. Some times qemu just dies after printing that message:

Quoted 7 lines

"Guest moved used index from 20151 to 1"

Some times it doesn't die, but I'm unable to log into the guest: I type username& password but the terminal kind of locks (the shell doesn't run).

Some times it works...

Here's the tail-end of the trace...

virtio_queue_notify 237.880 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40 virtqueue_pop 3.701 vq=0x7f11f4cb4d40 elem=0x7f11f5966118 in_num=0x2 out_num=0x1 virtio_blk_rw_complete 51.613 req=0x7f11f5966110 ret=0x0 virtio_blk_req_complete 1.327 req=0x7f11f5966110 status=0x0 virtqueue_fill 1.187 vq=0x7f11f4cb4d40 elem=0x7f11f5966118 len=0x1001 idx=0x0 virtqueue_flush 1.746 vq=0x7f11f4cb4d40 count=0x1
virtio_notify 1.537 vdev=0x7f11f4c8f5e0 vq=0x7f11f4cb4d40 virtio_queue_notify 374.978 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40 virtqueue_pop 3.631 vq=0x7f11f4cb4d40 elem=0x7f11f5faec58 in_num=0x2 out_num=0x1 virtio_blk_rw_complete 49.029 req=0x7f11f5faec50 ret=0x0 virtio_blk_req_complete 1.327 req=0x7f11f5faec50 status=0x0 virtqueue_fill 1.327 vq=0x7f11f4cb4d40 elem=0x7f11f5faec58 len=0x1001 idx=0x0 virtqueue_flush 1.746 vq=0x7f11f4cb4d40 count=0x1
virtio_notify 1.397 vdev=0x7f11f4c8f5e0 vq=0x7f11f4cb4d40 virtio_queue_notify 245.073 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40 virtqueue_pop 3.631 vq=0x7f11f4cb4d40 elem=0x7f11f5966118 in_num=0x2 out_num=0x1 virtio_blk_rw_complete 47.702 req=0x7f11f5966110 ret=0x0 virtio_blk_req_complete 1.257 req=0x7f11f5966110 status=0x0 virtqueue_fill 1.327 vq=0x7f11f4cb4d40 elem=0x7f11f5966118 len=0x1001 idx=0x0 virtqueue_flush 1.816 vq=0x7f11f4cb4d40 count=0x1
virtio_notify 1.327 vdev=0x7f11f4c8f5e0 vq=0x7f11f4cb4d40 virtio_queue_notify 450.616 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40 virtqueue_pop 4.051 vq=0x7f11f4cb4d40 elem=0x7f11f5faec58 in_num=0x2 out_num=0x1 virtio_blk_rw_complete 67.885 req=0x7f11f5faec50 ret=0x0 virtio_blk_req_complete 1.327 req=0x7f11f5faec50 status=0x0 virtqueue_fill 1.327 vq=0x7f11f4cb4d40 elem=0x7f11f5faec58 len=0x1001 idx=0x0

...
virtio_queue_notify 374.978 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40 virtqueue_pop 3.631 vq=0x7f11f4cb4d40 elem=0x7f11f5faec58 in_num=0x2 out_num=0x1 virtio_blk_rw_complete 49.029 req=0x7f11f5faec50 ret=0x0 virtio_blk_req_complete 1.327 req=0x7f11f5faec50 status=0x0 virtqueue_fill 1.327 vq=0x7f11f4cb4d40 elem=0x7f11f5faec58 len=0x1001 idx=0x0 virtqueue_flush 1.746 vq=0x7f11f4cb4d40 count=0x1
virtio_notify 1.397 vdev=0x7f11f4c8f5e0 vq=0x7f11f4cb4d40 virtio_queue_notify 245.073 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40 virtqueue_pop 3.631 vq=0x7f11f4cb4d40 elem=0x7f11f5966118 in_num=0x2 out_num=0x1 virtio_blk_rw_complete 47.702 req=0x7f11f5966110 ret=0x0 virtio_blk_req_complete 1.257 req=0x7f11f5966110 status=0x0 virtqueue_fill 1.327 vq=0x7f11f4cb4d40 elem=0x7f11f5966118 len=0x1001 idx=0x0 virtqueue_flush 1.816 vq=0x7f11f4cb4d40 count=0x1
virtio_notify 1.327 vdev=0x7f11f4c8f5e0 vq=0x7f11f4cb4d40 virtio_queue_notify 450.616 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40 virtqueue_pop 4.051 vq=0x7f11f4cb4d40 elem=0x7f11f5faec58 in_num=0x2 out_num=0x1 virtio_blk_rw_complete 67.885 req=0x7f11f5faec50 ret=0x0 virtio_blk_req_complete 1.327 req=0x7f11f5faec50 status=0x0 virtqueue_fill 1.327 vq=0x7f11f4cb4d40 elem=0x7f11f5faec58 len=0x1001 idx=0x0 virtqueue_flush 1.607 vq=0x7f11f4cb4d40 count=0x1
virtio_notify 1.257 vdev=0x7f11f4c8f5e0 vq=0x7f11f4cb4d40 virtio_queue_notify 196.813 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40 virtqueue_pop 3.562 vq=0x7f11f4cb4d40 elem=0x7f11f5966118 in_num=0x2 out_num=0x1 virtio_blk_rw_complete 47.492 req=0x7f11f5966110 ret=0x0 virtio_blk_req_complete 1.327 req=0x7f11f5966110 status=0x0 virtqueue_fill 1.327 vq=0x7f11f4cb4d40 elem=0x7f11f5966118 len=0x1001 idx=0x0 virtqueue_flush 1.676 vq=0x7f11f4cb4d40 count=0x1
virtio_notify 1.397 vdev=0x7f11f4c8f5e0 vq=0x7f11f4cb4d40 virtio_queue_notify 882289.570 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40

It doesn't seem to tell us much...but there's a bunch of successful reads before the final virtio_queue_notify, and that notify takes quite a bit longer than the previous ones. I can only speculate at this point, but I would guess this is when the guest has completed loading the saved memory from disk and it attempting to restore the previous state..

In the kernel there's a virtio_pci_suspend() PM callback that seems to get called around this time and restores the PCI config from virtio_pci_resume(). Could that be switching us to an older vring and throwing the QEMU side out of whack?

Quoted 53 lines
  qapi-schema-guest.json     |   11 +++++++++++
  qga/guest-agent-commands.c |   19 +++++++++++++++++++
  2 files changed, 30 insertions(+), 0 deletions(-)
qapi-schema-guest.json +11 -0
index fde5971..2c5bbcf 100644
@@ -215,3 +215,14 @@
  ##
  { 'command': 'guest-fsfreeze-thaw',
    'returns': 'int' }
+
+##
+# @guest-hibernate
+#
+# Save RAM contents to disk and powerdown the guest.
+#
+# Notes: This command doesn't return on success.
+#
+# Since: 1.1
+##
+{ 'command': 'guest-hibernate' }
qga/guest-agent-commands.c +19 -0
index 6da9904..9dd4060 100644
@@ -550,6 +550,25 @@ int64_t qmp_guest_fsfreeze_thaw(Error **err)
  }
  #endif
 
+#define LINUX_SYS_STATE_FILE "/sys/power/state"
+
+void qmp_guest_hibernate(Error **err)
+{
+    int fd;
+
+    fd = open(LINUX_SYS_STATE_FILE, O_WRONLY);
+    if (fd<  0) {
+        error_set(err, QERR_OPEN_FILE_FAILED, LINUX_SYS_STATE_FILE);
+        return;
+    }
+
+    if (write(fd, "disk", 4)<  0) {
+        error_set(err, QERR_UNDEFINED_ERROR);
+    }
+
+    close(fd);
+}
+
  /* register init/cleanup routines for stateful command groups */
  void ga_command_state_init(GAState *s, GACommandState *cs)
  {
#7 lore raw
Quoted 102 lines · On Thu, 08 Dec 2011 21:18:00 -0600 Michael Roth <[email protected]>…
Quoted 11 lines · Luiz Capitulino

This is basically suspend to disk on a Linux guest.

---

This is an RFC because I did it as simple as possible and I'm open to suggestions...

Now, while testing this or even "echo disk> /sys/power/state" I get several funny results. Some times qemu just dies after printing that message:

Quoted 7 lines

"Guest moved used index from 20151 to 1"

Some times it doesn't die, but I'm unable to log into the guest: I type username& password but the terminal kind of locks (the shell doesn't run).

Some times it works...

Here's the tail-end of the trace...

virtio_queue_notify 237.880 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40 virtqueue_pop 3.701 vq=0x7f11f4cb4d40 elem=0x7f11f5966118 in_num=0x2 out_num=0x1
virtio_blk_rw_complete 51.613 req=0x7f11f5966110 ret=0x0 virtio_blk_req_complete 1.327 req=0x7f11f5966110 status=0x0 virtqueue_fill 1.187 vq=0x7f11f4cb4d40 elem=0x7f11f5966118 len=0x1001 idx=0x0
virtqueue_flush 1.746 vq=0x7f11f4cb4d40 count=0x1 virtio_notify 1.537 vdev=0x7f11f4c8f5e0 vq=0x7f11f4cb4d40 virtio_queue_notify 374.978 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40 virtqueue_pop 3.631 vq=0x7f11f4cb4d40 elem=0x7f11f5faec58 in_num=0x2 out_num=0x1
virtio_blk_rw_complete 49.029 req=0x7f11f5faec50 ret=0x0 virtio_blk_req_complete 1.327 req=0x7f11f5faec50 status=0x0 virtqueue_fill 1.327 vq=0x7f11f4cb4d40 elem=0x7f11f5faec58 len=0x1001 idx=0x0
virtqueue_flush 1.746 vq=0x7f11f4cb4d40 count=0x1 virtio_notify 1.397 vdev=0x7f11f4c8f5e0 vq=0x7f11f4cb4d40 virtio_queue_notify 245.073 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40 virtqueue_pop 3.631 vq=0x7f11f4cb4d40 elem=0x7f11f5966118 in_num=0x2 out_num=0x1
virtio_blk_rw_complete 47.702 req=0x7f11f5966110 ret=0x0 virtio_blk_req_complete 1.257 req=0x7f11f5966110 status=0x0 virtqueue_fill 1.327 vq=0x7f11f4cb4d40 elem=0x7f11f5966118 len=0x1001 idx=0x0
virtqueue_flush 1.816 vq=0x7f11f4cb4d40 count=0x1 virtio_notify 1.327 vdev=0x7f11f4c8f5e0 vq=0x7f11f4cb4d40 virtio_queue_notify 450.616 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40 virtqueue_pop 4.051 vq=0x7f11f4cb4d40 elem=0x7f11f5faec58 in_num=0x2 out_num=0x1
virtio_blk_rw_complete 67.885 req=0x7f11f5faec50 ret=0x0 virtio_blk_req_complete 1.327 req=0x7f11f5faec50 status=0x0 virtqueue_fill 1.327 vq=0x7f11f4cb4d40 elem=0x7f11f5faec58 len=0x1001 idx=0x0

...
virtio_queue_notify 374.978 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40 virtqueue_pop 3.631 vq=0x7f11f4cb4d40 elem=0x7f11f5faec58 in_num=0x2 out_num=0x1
virtio_blk_rw_complete 49.029 req=0x7f11f5faec50 ret=0x0 virtio_blk_req_complete 1.327 req=0x7f11f5faec50 status=0x0 virtqueue_fill 1.327 vq=0x7f11f4cb4d40 elem=0x7f11f5faec58 len=0x1001 idx=0x0
virtqueue_flush 1.746 vq=0x7f11f4cb4d40 count=0x1 virtio_notify 1.397 vdev=0x7f11f4c8f5e0 vq=0x7f11f4cb4d40 virtio_queue_notify 245.073 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40 virtqueue_pop 3.631 vq=0x7f11f4cb4d40 elem=0x7f11f5966118 in_num=0x2 out_num=0x1
virtio_blk_rw_complete 47.702 req=0x7f11f5966110 ret=0x0 virtio_blk_req_complete 1.257 req=0x7f11f5966110 status=0x0 virtqueue_fill 1.327 vq=0x7f11f4cb4d40 elem=0x7f11f5966118 len=0x1001 idx=0x0
virtqueue_flush 1.816 vq=0x7f11f4cb4d40 count=0x1 virtio_notify 1.327 vdev=0x7f11f4c8f5e0 vq=0x7f11f4cb4d40 virtio_queue_notify 450.616 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40 virtqueue_pop 4.051 vq=0x7f11f4cb4d40 elem=0x7f11f5faec58 in_num=0x2 out_num=0x1
virtio_blk_rw_complete 67.885 req=0x7f11f5faec50 ret=0x0 virtio_blk_req_complete 1.327 req=0x7f11f5faec50 status=0x0 virtqueue_fill 1.327 vq=0x7f11f4cb4d40 elem=0x7f11f5faec58 len=0x1001 idx=0x0
virtqueue_flush 1.607 vq=0x7f11f4cb4d40 count=0x1 virtio_notify 1.257 vdev=0x7f11f4c8f5e0 vq=0x7f11f4cb4d40 virtio_queue_notify 196.813 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40 virtqueue_pop 3.562 vq=0x7f11f4cb4d40 elem=0x7f11f5966118 in_num=0x2 out_num=0x1
virtio_blk_rw_complete 47.492 req=0x7f11f5966110 ret=0x0 virtio_blk_req_complete 1.327 req=0x7f11f5966110 status=0x0 virtqueue_fill 1.327 vq=0x7f11f4cb4d40 elem=0x7f11f5966118 len=0x1001 idx=0x0
virtqueue_flush 1.676 vq=0x7f11f4cb4d40 count=0x1 virtio_notify 1.397 vdev=0x7f11f4c8f5e0 vq=0x7f11f4cb4d40 virtio_queue_notify 882289.570 vdev=0x7f11f4c8f5e0 n=0x0 vq=0x7f11f4cb4d40

It doesn't seem to tell us much...but there's a bunch of successful reads before the final virtio_queue_notify, and that notify takes quite a bit longer than the previous ones. I can only speculate at this point, but I would guess this is when the guest has completed loading the saved memory from disk and it attempting to restore the previous state..

Quoted 4 lines

In the kernel there's a virtio_pci_suspend() PM callback that seems to get called around this time and restores the PCI config from virtio_pci_resume(). Could that be switching us to an older vring and throwing the QEMU side out of whack?

Not sure. But Amit has confirmed that it's a virtio bug and he's working on it. Right Amit?

Quoted 56 lines
Quoted 55 lines

  qapi-schema-guest.json     |   11 +++++++++++
  qga/guest-agent-commands.c |   19 +++++++++++++++++++
  2 files changed, 30 insertions(+), 0 deletions(-)
qapi-schema-guest.json +11 -0
index fde5971..2c5bbcf 100644
@@ -215,3 +215,14 @@
  ##
  { 'command': 'guest-fsfreeze-thaw',
    'returns': 'int' }
+
+##
+# @guest-hibernate
+#
+# Save RAM contents to disk and powerdown the guest.
+#
+# Notes: This command doesn't return on success.
+#
+# Since: 1.1
+##
+{ 'command': 'guest-hibernate' }
qga/guest-agent-commands.c +19 -0
index 6da9904..9dd4060 100644
@@ -550,6 +550,25 @@ int64_t qmp_guest_fsfreeze_thaw(Error **err)
  }
  #endif
 
+#define LINUX_SYS_STATE_FILE "/sys/power/state"
+
+void qmp_guest_hibernate(Error **err)
+{
+    int fd;
+
+    fd = open(LINUX_SYS_STATE_FILE, O_WRONLY);
+    if (fd<  0) {
+        error_set(err, QERR_OPEN_FILE_FAILED, LINUX_SYS_STATE_FILE);
+        return;
+    }
+
+    if (write(fd, "disk", 4)<  0) {
+        error_set(err, QERR_UNDEFINED_ERROR);
+    }
+
+    close(fd);
+}
+
  /* register init/cleanup routines for stateful command groups */
  void ga_command_state_init(GAState *s, GACommandState *cs)
  {
#8 lore raw
Quoted 21 lines · Luiz Capitulino
Quoted 18 lines · On Thu, 08 Dec 2011 21:18:00 -0600 Michael Roth <[email protected]>…
Quoted 17 lines · Luiz Capitulino

This is basically suspend to disk on a Linux guest.

---

This is an RFC because I did it as simple as possible and I'm open to suggestions...

Now, while testing this or even "echo disk> /sys/power/state" I get several funny results. Some times qemu just dies after printing that message:

"Guest moved used index from 20151 to 1"

Some times it doesn't die, but I'm unable to log into the guest: I type username& password but the terminal kind of locks (the shell doesn't run).

Some times it works...

...

Quoted 6 lines
Quoted 6 lines

It doesn't seem to tell us much...but there's a bunch of successful reads before the final virtio_queue_notify, and that notify takes quite a bit longer than the previous ones. I can only speculate at this point, but I would guess this is when the guest has completed loading the saved memory from disk and it attempting to restore the previous state..

Quoted 7 lines
Quoted 4 lines

In the kernel there's a virtio_pci_suspend() PM callback that seems to get called around this time and restores the PCI config from virtio_pci_resume(). Could that be switching us to an older vring and throwing the QEMU side out of whack?

Not sure. But Amit has confirmed that it's a virtio bug and he's working on it. Right Amit?

Yes. You can try the series at

https://lkml.org/lkml/2011/12/6/479

in the guest kernel to see it all work.

Amit

#9 lore raw
In reply to Luiz Capitulino
Quoted 5 lines · Luiz Capitulino

This is basically suspend to disk on a Linux guest.

---

Quoted 7 lines

This is an RFC because I did it as simple as possible and I'm open to suggestions...

Now, while testing this or even "echo disk > /sys/power/state" I get several funny results. Some times qemu just dies after printing that message:

"Guest moved used index from 20151 to 1"

Virtio drivers don't handle S4 at all (yet). I have patches that are being discussed to add this support. Please try without virtio devices.

Quoted 2 lines

Some times it doesn't die, but I'm unable to log into the guest: I type username & password but the terminal kind of locks (the shell doesn't run).

kvmclock too doesn't handle resume from S4 yet. So use '-kvmclock' for your cpu definition (or clock=pmtmr in your kernel cmd line).

Thanks for the patch!

Amit

#10 lore raw
In reply to Luiz Capitulino
Quoted 9 lines · Luiz Capitulino

This is basically suspend to disk on a Linux guest.

---

This is an RFC because I did it as simple as possible and I'm open to suggestions...

Now, while testing this or even "echo disk> /sys/power/state" I get several

Beyond the previous comments about virtio s4 that Amit solved and pm-hibernate I think it will be nice to add suspend to ram (s3) option too. Currently qemu wakes up immediately after s3 but it can be fixed and it can pause the guest and wait to a keyboard event or monitor command.

Quoted 62 lines

funny results. Some times qemu just dies after printing that message:

"Guest moved used index from 20151 to 1"

Some times it doesn't die, but I'm unable to log into the guest: I type username& password but the terminal kind of locks (the shell doesn't run).

Some times it works...

  qapi-schema-guest.json     |   11 +++++++++++
  qga/guest-agent-commands.c |   19 +++++++++++++++++++
  2 files changed, 30 insertions(+), 0 deletions(-)
qapi-schema-guest.json +11 -0
index fde5971..2c5bbcf 100644
@@ -215,3 +215,14 @@
  ##
  { 'command': 'guest-fsfreeze-thaw',
    'returns': 'int' }
+
+##
+# @guest-hibernate
+#
+# Save RAM contents to disk and powerdown the guest.
+#
+# Notes: This command doesn't return on success.
+#
+# Since: 1.1
+##
+{ 'command': 'guest-hibernate' }
qga/guest-agent-commands.c +19 -0
index 6da9904..9dd4060 100644
@@ -550,6 +550,25 @@ int64_t qmp_guest_fsfreeze_thaw(Error **err)
  }
  #endif
 
+#define LINUX_SYS_STATE_FILE "/sys/power/state"
+
+void qmp_guest_hibernate(Error **err)
+{
+    int fd;
+
+    fd = open(LINUX_SYS_STATE_FILE, O_WRONLY);
+    if (fd<  0) {
+        error_set(err, QERR_OPEN_FILE_FAILED, LINUX_SYS_STATE_FILE);
+        return;
+    }
+
+    if (write(fd, "disk", 4)<  0) {
+        error_set(err, QERR_UNDEFINED_ERROR);
+    }
+
+    close(fd);
+}
+
  /* register init/cleanup routines for stateful command groups */
  void ga_command_state_init(GAState *s, GACommandState *cs)
  {
#11 lore raw
Quoted 9 lines · On Sun, 11 Dec 2011 12:00:12 +0200 Dor Laor <[email protected]> wrote:
Quoted 8 lines · Luiz Capitulino

This is basically suspend to disk on a Linux guest.

---

This is an RFC because I did it as simple as possible and I'm open to suggestions...

Quoted 6 lines
Quoted 1 line

Now, while testing this or even "echo disk> /sys/power/state" I get several

Beyond the previous comments about virtio s4 that Amit solved and pm-hibernate I think it will be nice to add suspend to ram (s3) option too. Currently qemu wakes up immediately after s3 but it can be fixed and it can pause the guest and wait to a keyboard event or monitor command.

I thought adding it in this RFC, but I decided to keep it simple. Will add it to v2.

Quoted 64 lines
Quoted 63 lines

funny results. Some times qemu just dies after printing that message:

"Guest moved used index from 20151 to 1"

Some times it doesn't die, but I'm unable to log into the guest: I type username& password but the terminal kind of locks (the shell doesn't run).

Some times it works...

  qapi-schema-guest.json     |   11 +++++++++++
  qga/guest-agent-commands.c |   19 +++++++++++++++++++
  2 files changed, 30 insertions(+), 0 deletions(-)
qapi-schema-guest.json +11 -0
index fde5971..2c5bbcf 100644
@@ -215,3 +215,14 @@
  ##
  { 'command': 'guest-fsfreeze-thaw',
    'returns': 'int' }
+
+##
+# @guest-hibernate
+#
+# Save RAM contents to disk and powerdown the guest.
+#
+# Notes: This command doesn't return on success.
+#
+# Since: 1.1
+##
+{ 'command': 'guest-hibernate' }
qga/guest-agent-commands.c +19 -0
index 6da9904..9dd4060 100644
@@ -550,6 +550,25 @@ int64_t qmp_guest_fsfreeze_thaw(Error **err)
  }
  #endif
 
+#define LINUX_SYS_STATE_FILE "/sys/power/state"
+
+void qmp_guest_hibernate(Error **err)
+{
+    int fd;
+
+    fd = open(LINUX_SYS_STATE_FILE, O_WRONLY);
+    if (fd<  0) {
+        error_set(err, QERR_OPEN_FILE_FAILED, LINUX_SYS_STATE_FILE);
+        return;
+    }
+
+    if (write(fd, "disk", 4)<  0) {
+        error_set(err, QERR_UNDEFINED_ERROR);
+    }
+
+    close(fd);
+}
+
  /* register init/cleanup routines for stateful command groups */
  void ga_command_state_init(GAState *s, GACommandState *cs)
  {