Skip to contentSkip to conversation

xen_disk: Disable file locking for the PV disk backend

PATCH In review Anthony PERARD 3 participants 6 messages May 22, 2019 – Jun 14, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xen_disk: Disable file locking for the PV disk backend

Since QEMU 2.10 (or qemu-xen-4.10), qemu locks disk images to avoid them been re-opened in a different qemu process.

With Xen, there are two issues:
- For HVM guests, a disk image can be open twice! One by the emulation driver, and one by the PV backend.
- During migration, the qemu process of the newly spawned domain may attempt to access the disk image before the domain been migrated and the qemu process are been completely destroyed.

Migration of HVM guest as been taken care of in libxl, but migration of PV guest with qdisk and HVM guest attempting to access the PV disk before unplugging the emulated disk are still an issue.

For these reasons, we don't want to have QEMU use a locking mechanism with the PV backend.

This is already done by db9ff46eeb in QEMU upstream, or QEMU 4.0.

Affected version of QEMU are:
- qemu-xen of Xen 4.10 and 4.11
- QEMU 2.10, 2.11, 2.12, 3.0 and 3.1

---

Hi Olaf, Roger, Wei,

Instead of a fix in libxl (with the reverted patch "libxl: fix migration of PV and PVH domUs with and without qemu"), what do you think of this QEMU patch? It is much simpler that trying to work around the issue in libxl, and it fix one more problem that still exist.

This patch is only for stable branches as Xen unstable already have a qemu with a similar patch.

Can you give it a try with one of the affected qemu? (qemu-xen-4.10 or qemu-xen-4.11)

Thanks,

---
 hw/block/xen_disk.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
hw/block/xen_disk.c +8 -1
index e431bd89e8..cbff174a98 100644
@@ -1084,11 +1084,18 @@ static int blk_connect(struct XenDevice *xendev)
         Error *local_err = NULL;
         QDict *options = NULL;
 
+        options = qdict_new();
+
         if (strcmp(blkdev->fileproto, "<unset>")) {
-            options = qdict_new();
             qdict_put_str(options, "driver", blkdev->fileproto);
         }
 
+        /*
+         * It is necessary to turn file locking off as an emulated device
+         * may have already opened the same image file.
+         */
+        qdict_put_str(options, "file.locking", "off");
+
         /* setup via xenbus -> create new block driver instance */
         xen_pv_printf(&blkdev->xendev, 2, "create new bdrv (xenbus setup)\n");
         blkdev->blk = blk_new_open(blkdev->filename, NULL, options,
Signature · 7 lines
Anthony PERARD


_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#1 lore raw
Quoted 35 lines · Anthony PERARD

Since QEMU 2.10 (or qemu-xen-4.10), qemu locks disk images to avoid them been re-opened in a different qemu process.

With Xen, there are two issues:
- For HVM guests, a disk image can be open twice! One by the emulation driver, and one by the PV backend.
- During migration, the qemu process of the newly spawned domain may attempt to access the disk image before the domain been migrated and the qemu process are been completely destroyed.

Migration of HVM guest as been taken care of in libxl, but migration of PV guest with qdisk and HVM guest attempting to access the PV disk before unplugging the emulated disk are still an issue.

For these reasons, we don't want to have QEMU use a locking mechanism with the PV backend.

This is already done by db9ff46eeb in QEMU upstream, or QEMU 4.0.

Affected version of QEMU are:
- qemu-xen of Xen 4.10 and 4.11
- QEMU 2.10, 2.11, 2.12, 3.0 and 3.1

---

Hi Olaf, Roger, Wei,

Instead of a fix in libxl (with the reverted patch "libxl: fix migration of PV and PVH domUs with and without qemu"), what do you think of this QEMU patch? It is much simpler that trying to work around the issue in libxl, and it fix one more problem that still exist.

Quoted 7 lines

This patch is only for stable branches as Xen unstable already have a qemu with a similar patch.

Can you give it a try with one of the affected qemu? (qemu-xen-4.10 or qemu-xen-4.11)

Thanks,

Thanks for stepping up. :-)

Quoted 31 lines
---
 hw/block/xen_disk.c | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
hw/block/xen_disk.c +8 -1
index e431bd89e8..cbff174a98 100644
@@ -1084,11 +1084,18 @@ static int blk_connect(struct XenDevice *xendev)
         Error *local_err = NULL;
         QDict *options = NULL;
 
+        options = qdict_new();
+
         if (strcmp(blkdev->fileproto, "<unset>")) {
-            options = qdict_new();
             qdict_put_str(options, "driver", blkdev->fileproto);
         }
 
+        /*
+         * It is necessary to turn file locking off as an emulated device
+         * may have already opened the same image file.
+         */
+        qdict_put_str(options, "file.locking", "off");
+
         /* setup via xenbus -> create new block driver instance */
         xen_pv_printf(&blkdev->xendev, 2, "create new bdrv (xenbus setup)\n");
         blkdev->blk = blk_new_open(blkdev->filename, NULL, options,
Signature · 1 line
Anthony PERARD

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#2 lore raw
In reply to Anthony PERARD

Am Wed, 22 May 2019 15:51:40 +0100
schrieb Anthony PERARD <[email protected]>:

Quoted 2 lines

Can you give it a try with one of the affected qemu? (qemu-xen-4.10 or qemu-xen-4.11)

Thanks for the patch. Unfortunately there is no easy way to trigger the race. Is the changed code path also exercised for PV domUs?

Olaf

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#3 lore raw
Quoted 8 lines · Olaf Hering

Am Wed, 22 May 2019 15:51:40 +0100
schrieb Anthony PERARD <[email protected]>:

Quoted 2 lines

Can you give it a try with one of the affected qemu? (qemu-xen-4.10 or qemu-xen-4.11)

Thanks for the patch. Unfortunately there is no easy way to trigger the race. Is the changed code path also exercised for PV domUs?

Yes! The code path is executed every time a disk is attached via the qdisk backend. So as long as guest can access the pv disk, then it's fine (for both HVM and PV domUs, and PVH).

Another way to make sure that the locking is disable is to attach a disk to a second guest, qemu should not complain anymore. I've tested both "raw" disk and "qcow2" disk but I'm not sure if other kind of disk needs to be tested.

Thanks,

Signature · 6 lines
Anthony PERARD

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#4 lore raw
Quoted 10 lines · Anthony PERARD
Quoted 8 lines · Olaf Hering

Am Wed, 22 May 2019 15:51:40 +0100
schrieb Anthony PERARD <[email protected]>:

Quoted 2 lines

Can you give it a try with one of the affected qemu? (qemu-xen-4.10 or qemu-xen-4.11)

Thanks for the patch. Unfortunately there is no easy way to trigger the race. Is the changed code path also exercised for PV domUs?

Quoted 8 lines

Yes! The code path is executed every time a disk is attached via the qdisk backend. So as long as guest can access the pv disk, then it's fine (for both HVM and PV domUs, and PVH).

Another way to make sure that the locking is disable is to attach a disk to a second guest, qemu should not complain anymore. I've tested both "raw" disk and "qcow2" disk but I'm not sure if other kind of disk needs to be tested.

Olaf, did you apply this patch and run some guest? Do they still boot? It doesn't matter if you can't trigger the race with or without this patch, I just want to know if there are any regression.
So, may have I your "Tested-by" ? And I will try to apply it to our qemu-xen branches.

Thanks,

Signature · 6 lines
Anthony PERARD

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
#5 lore raw

Am Wed, 5 Jun 2019 11:53:38 +0100
schrieb Anthony PERARD <[email protected]>:

Quoted 5 lines

Olaf, did you apply this patch and run some guest? Do they still boot? It doesn't matter if you can't trigger the race with or without this patch, I just want to know if there are any regression.
So, may have I your "Tested-by" ? And I will try to apply it to our qemu-xen branches.

I can not reproduce it myself on the systems I have available. It seems that patch went already into qemu-xen.git.

Olaf

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel