Skip to contentSkip to conversation

libxl: always call qemus xen-save-devices-state in suspend/resume

PATCH In review Olaf Hering 5 participants 13 messages May 17, 2018 – Jun 21, 2018

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 libxl: always call qemus xen-save-devices-state in suspend/resume

If a domU has a qemu-xen instance attached, it is required to call qemus "xen-save-devices-state" method. Without it, the receiving side of a PV migration may be unable to lock the image:

xen be: qdisk-51712: xen be: qdisk-51712: error: Failed to get "write" lock
error: Failed to get "write" lock
xen be: qdisk-51712: xen be: qdisk-51712: initialise() failed initialise() failed

The proper way to decide if a PV or PVH domU has a qemu-xen running is to use the libxl__need_xenpv_qemu API. But since there is no copy of a libxl_domain_config available in these places, it should be enough to check if xenstore contains the relevant info.

---
 tools/libxl/libxl_dom_suspend.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
tools/libxl/libxl_dom_suspend.c +6 -2
index 1e904bae8a..0d769eefd8 100644
@@ -377,7 +377,9 @@ static void domain_suspend_common_guest_suspended(libxl__egc *egc,
     libxl__ev_xswatch_deregister(gc, &dsps->guest_watch);
     libxl__ev_time_deregister(gc, &dsps->guest_timeout);
 
-    if (dsps->type == LIBXL_DOMAIN_TYPE_HVM) {
+    if (dsps->type == LIBXL_DOMAIN_TYPE_HVM ||
+        libxl__device_model_version_running(gc, dsps->domid) ==
+        LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN) {
         rc = libxl__domain_suspend_device_model(gc, dsps);
         if (rc) {
             LOGD(ERROR, dsps->domid,
@@ -460,7 +462,9 @@ int libxl__domain_resume(libxl__gc *gc, uint32_t domid, int suspend_cancel)
         goto out;
     }
 
-    if (type == LIBXL_DOMAIN_TYPE_HVM) {
+    if (type == LIBXL_DOMAIN_TYPE_HVM ||
+        libxl__device_model_version_running(gc, domid) ==
+        LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN) {
         rc = libxl__domain_resume_device_model(gc, domid);
         if (rc) {
             LOGD(ERROR, domid, "failed to resume device model:%d", rc);

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#1 lore raw
Quoted 9 lines · Olaf Hering

If a domU has a qemu-xen instance attached, it is required to call qemus "xen-save-devices-state" method. Without it, the receiving side of a PV migration may be unable to lock the image:

xen be: qdisk-51712: xen be: qdisk-51712: error: Failed to get "write" lock
error: Failed to get "write" lock
xen be: qdisk-51712: xen be: qdisk-51712: initialise() failed initialise() failed

Quoted 6 lines

The proper way to decide if a PV or PVH domU has a qemu-xen running is to use the libxl__need_xenpv_qemu API. But since there is no copy of a libxl_domain_config available in these places, it should be enough to check if xenstore contains the relevant info.

I'm rather sure this patch is repairing the following:

Tests which are failing intermittently (not blocking):
test-amd64-amd64-xl-qemut-debianhvm-amd64 16 guest-localmigrate/x10
fail pass in 122715
test-amd64-amd64-xl-qemuu-debianhvm-amd64 16 guest-localmigrate/x10
fail pass in 122715

In case the maintainers are fine with this patch you can add my:

Juergen

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#3 lore raw
In reply to Juergen Gross
Quoted 17 lines · Juergen Gross
Quoted 9 lines · Olaf Hering

If a domU has a qemu-xen instance attached, it is required to call qemus "xen-save-devices-state" method. Without it, the receiving side of a PV migration may be unable to lock the image:

xen be: qdisk-51712: xen be: qdisk-51712: error: Failed to get "write" lock
error: Failed to get "write" lock
xen be: qdisk-51712: xen be: qdisk-51712: initialise() failed initialise() failed

Quoted 6 lines

The proper way to decide if a PV or PVH domU has a qemu-xen running is to use the libxl__need_xenpv_qemu API. But since there is no copy of a libxl_domain_config available in these places, it should be enough to check if xenstore contains the relevant info.

Quoted 7 lines

I'm rather sure this patch is repairing the following:

Tests which are failing intermittently (not blocking):
test-amd64-amd64-xl-qemut-debianhvm-amd64 16 guest-localmigrate/x10
fail pass in 122715
test-amd64-amd64-xl-qemuu-debianhvm-amd64 16 guest-localmigrate/x10
fail pass in 122715

I don't think so. The locking issue is known and fixed (at least for HVM). It would cause local migration to fail definitively, while the log says those two tests fail intermittently.

Wei.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#4 lore raw
In reply to Olaf Hering

Cc Anthony.

Quoted 9 lines · Olaf Hering

If a domU has a qemu-xen instance attached, it is required to call qemus "xen-save-devices-state" method. Without it, the receiving side of a PV migration may be unable to lock the image:

xen be: qdisk-51712: xen be: qdisk-51712: error: Failed to get "write" lock
error: Failed to get "write" lock
xen be: qdisk-51712: xen be: qdisk-51712: initialise() failed initialise() failed

The commit message can use a bit of improvement. Strictly speaking, this is only a problem when qdisk is in use.

But I think it should be harmless to issue that command anyway.

Quoted 15 lines

The proper way to decide if a PV or PVH domU has a qemu-xen running is to use the libxl__need_xenpv_qemu API. But since there is no copy of a libxl_domain_config available in these places, it should be enough to check if xenstore contains the relevant info.

---
 tools/libxl/libxl_dom_suspend.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
tools/libxl/libxl_dom_suspend.c +0 -0
index 1e904bae8a..0d769eefd8 100644

tools/libxl/libxl_dom_suspend.c static void domain_suspend_common_guest_suspended(libxl__egc *egc, · line 382

Quoted 7 lines
tools/libxl/libxl_dom_suspend.c +3 -1
@@ -377,7 +377,9 @@ static void domain_suspend_common_guest_suspended(libxl__egc *egc,
     libxl__ev_xswatch_deregister(gc, &dsps->guest_watch);
     libxl__ev_time_deregister(gc, &dsps->guest_timeout);
 
-    if (dsps->type == LIBXL_DOMAIN_TYPE_HVM) {
+    if (dsps->type == LIBXL_DOMAIN_TYPE_HVM ||
+        libxl__device_model_version_running(gc, dsps->domid) ==
+        LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN) {

You don't need to test if the guest is HVM anymore. You only need to know if QEMU upstream is running.

Anthony, please check if I've said anything wrong.

Wei.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#5 lore raw

Am Fri, 18 May 2018 18:01:42 +0100
schrieb Wei Liu <[email protected]>:

Quoted 2 lines

You don't need to test if the guest is HVM anymore. You only need to know if QEMU upstream is running.

libxl__domain_suspend_device_model handles qemu-xen and qemu-xen-trad. That function can not be called unconditionally I think. Perhaps I should adjust the commit message.

Olaf

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#6 lore raw
Quoted 9 lines · Olaf Hering

Am Fri, 18 May 2018 18:01:42 +0100
schrieb Wei Liu <[email protected]>:

Quoted 2 lines

You don't need to test if the guest is HVM anymore. You only need to know if QEMU upstream is running.

libxl__domain_suspend_device_model handles qemu-xen and qemu-xen-trad. That function can not be called unconditionally I think. Perhaps I should adjust the commit message.

What I mean is: now you seem to want to call libxl__domain_suspend_device_model when QEMU upstream is running, regardless of whether a guest is HVM or PV, so the check for HVM should be gone.

Does that make sense?

Wei.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#7 lore raw

Am Mon, 21 May 2018 09:18:20 +0100
schrieb Wei Liu <[email protected]>:

Quoted 4 lines

What I mean is: now you seem to want to call libxl__domain_suspend_device_model when QEMU upstream is running, regardless of whether a guest is HVM or PV, so the check for HVM should be gone.

libxl__domain_suspend_device_model calls libxl__device_model_version_running, which may return LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL in case "dm-version" does not exist.
Is it valid to assume that a non-existant "dm-version" means qemu-trad? So far I have not tested a PV domU with qemu-trad, not sure if that can work anyway.

Also, libxl__device_model_version_running may return -1. Is that valid?

Olaf

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#8 lore raw
Quoted 8 lines · Olaf Hering

Am Mon, 21 May 2018 09:18:20 +0100
schrieb Wei Liu <[email protected]>:

Quoted 4 lines

What I mean is: now you seem to want to call libxl__domain_suspend_device_model when QEMU upstream is running, regardless of whether a guest is HVM or PV, so the check for HVM should be gone.

Quoted 9 lines

libxl__domain_suspend_device_model calls libxl__device_model_version_running, which may return LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL in case "dm-version" does not exist.
Is it valid to assume that a non-existant "dm-version" means qemu-trad? So far I have not tested a PV domU with qemu-trad, not sure if that can work anyway.

Also, libxl__device_model_version_running may return -1. Is that valid?

I think your predicate is correct. Sorry for the noise.

Wei.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#10 lore raw
In reply to Wei Liu
Quoted 44 lines · Wei Liu

Cc Anthony.

Quoted 9 lines · Olaf Hering

If a domU has a qemu-xen instance attached, it is required to call qemus "xen-save-devices-state" method. Without it, the receiving side of a PV migration may be unable to lock the image:

xen be: qdisk-51712: xen be: qdisk-51712: error: Failed to get "write" lock
error: Failed to get "write" lock
xen be: qdisk-51712: xen be: qdisk-51712: initialise() failed initialise() failed

The commit message can use a bit of improvement. Strictly speaking, this is only a problem when qdisk is in use.

But I think it should be harmless to issue that command anyway.

Quoted 15 lines

The proper way to decide if a PV or PVH domU has a qemu-xen running is to use the libxl__need_xenpv_qemu API. But since there is no copy of a libxl_domain_config available in these places, it should be enough to check if xenstore contains the relevant info.

---
 tools/libxl/libxl_dom_suspend.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
tools/libxl/libxl_dom_suspend.c +0 -0
index 1e904bae8a..0d769eefd8 100644

tools/libxl/libxl_dom_suspend.c static void domain_suspend_common_guest_suspended(libxl__egc *egc, · line 382

Quoted 7 lines
tools/libxl/libxl_dom_suspend.c +3 -1
@@ -377,7 +377,9 @@ static void domain_suspend_common_guest_suspended(libxl__egc *egc,
     libxl__ev_xswatch_deregister(gc, &dsps->guest_watch);
     libxl__ev_time_deregister(gc, &dsps->guest_timeout);
 
-    if (dsps->type == LIBXL_DOMAIN_TYPE_HVM) {
+    if (dsps->type == LIBXL_DOMAIN_TYPE_HVM ||
+        libxl__device_model_version_running(gc, dsps->domid) ==
+        LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN) {

You don't need to test if the guest is HVM anymore. You only need to know if QEMU upstream is running.

Quoted 1 line

Anthony, please check if I've said anything wrong.

I think the patch is fine.

The resume side is not needed for PV because that only resume emulation, and qemu start with emulation running, but it is armless to call the command.

Thanks,

Signature · 6 lines
Anthony PERARD

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#11 lore raw
In reply to Olaf Hering

Am Thu, 17 May 2018 17:51:08 +0200
schrieb Olaf Hering <[email protected]>:

tools/libxl/libxl_dom_suspend.c static void domain_suspend_common_guest_suspended(libxl__egc *egc, · line 378

Quoted 3 lines
tools/libxl/libxl_dom_suspend.c +2 -0
@@ -377,7 +377,9 @@ static void domain_suspend_common_guest_suspended(libxl__egc *egc,
+        libxl__device_model_version_running(gc, dsps->domid) ==
+        LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN) {

Please withdraw that patch. This condition is always true for PV.

Olaf

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#12 lore raw

Olaf Hering writes ("Re: [PATCH v1] libxl: always call qemus xen-save-devices-state in suspend/resume"):

Quoted 8 lines

Am Thu, 17 May 2018 17:51:08 +0200
schrieb Olaf Hering <[email protected]>:

tools/libxl/libxl_dom_suspend.c static void domain_suspend_common_guest_suspended(libxl__egc *egc, · line 378

Quoted 3 lines
tools/libxl/libxl_dom_suspend.c +2 -0
@@ -377,7 +377,9 @@ static void domain_suspend_common_guest_suspended(libxl__egc *egc,
+        libxl__device_model_version_running(gc, dsps->domid) ==
+        LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN) {

Please withdraw that patch. This condition is always true for PV.

OK.

Ian.

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel