unmodified_drivers: unplug the emulated devices at resume time

PATCH In review Olaf Hering 4 participants 6 messages Jun 12, 2018 – Jun 13, 2018

Share & tools

Olaf Hering

author

2018-06-12 14:10 UTC #0 lore raw

PATCH 1/1 unmodified_drivers: unplug the emulated devices at resume time

Since qemu-2.10 it is required to unplug emulated devices again after a live migration. If this is not done, qemu's block-backend driver will be unable to open the backing disk image because it is still busy by qemu's IDE driver. As a result the domUs block-frontend driver will be unable to access the disks, and the domU has to be destroyed. libxl is unable to detect the situation.

Apply the same workaround for this qemu bug that was done already years ago in linux.git with commit 512b109ec962 ("xen: unplug the emulated devices at resume time") to make sure xenlinux based domUs can be migrated to unfixed hosts.

Signed-off-by Olaf Hering <[email protected]>

---
 .../linux-2.6/platform-pci/platform-pci.c             | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

unmodified_drivers/linux-2.6/platform-pci/platform-pci.c +10 -1

index a7dbd0a8ab..8e4e814d29 100644
@@ -285,6 +285,12 @@ int gnttab_init(void);
 #define UNPLUG_AUX_IDE_DISKS 4
 #define UNPLUG_ALL 7
 
+static short unplug_value;
+static void unplug_devices(void)
+{
+	outw(unplug_value, XEN_IOPORT_UNPLUG);
+}
+
 static int check_platform_magic(struct device *dev, long ioaddr, long iolen)
 {
 	short magic, unplug = 0;

@@ -312,6 +318,7 @@ static int check_platform_magic(struct device *dev, long ioaddr, long iolen)
 			dev_warn(dev, "unrecognised option '%s' "
 				 "in module parameter 'dev_unplug'\n", p);
 	}
+	unplug_value = unplug;
 
 	if (iolen < 0x16) {
 		err = "backend too old";

@@ -339,7 +346,7 @@ static int check_platform_magic(struct device *dev, long ioaddr, long iolen)
 		}
 		/* Fall through */
 	case 0:
-		outw(unplug, XEN_IOPORT_UNPLUG);
+		unplug_devices();
 		break;
 	default:
 		err = "unknown I/O protocol version";

@@ -498,6 +505,8 @@ void platform_pci_resume(void)
 
 	if (set_callback_via(callback_via))
 		printk("platform_pci_resume failure!\n");
+
+	unplug_devices();
 }
 
 static int __init platform_pci_module_init(void)

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Jan Beulich

2018-06-12 15:01 UTC #1 lore raw

Quoted 7 lines

Quoted 1 line
Quoted 1 line · On 12.06.18 at 16:10, <[email protected]> wrote:
Since qemu-2.10 it is required to unplug emulated devices again after a live migration. If this is not done, qemu's block-backend driver will be unable to open the backing disk image because it is still busy by qemu's IDE driver. As a result the domUs block-frontend driver will be unable to access the disks, and the domU has to be destroyed. libxl is unable to detect the situation.

So there's no plan to fix this on the qemu side?

Quoted 4 lines

Apply the same workaround for this qemu bug that was done already years ago in linux.git with commit 512b109ec962 ("xen: unplug the emulated devices at resume time") to make sure xenlinux based domUs can be migrated to unfixed hosts.

Hmm, that change has no explanation whatsoever of why this was necessary already back at that time. Stefano?

Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Olaf Hering

author

2018-06-12 15:05 UTC #2 lore raw

Am Tue, 12 Jun 2018 09:01:07 -0600
schrieb "Jan Beulich" <[email protected]>:

Quoted 1 line

So there's no plan to fix this on the qemu side?

Even if there would be one, existing dom0s may not get an update in time.

I sent some prototype of a potential fix for qemu, it needs more work.

Olaf

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Stefano Stabellini

2018-06-12 19:39 UTC #3 lore raw

In reply to Jan Beulich

Quoted 10 lines · Jan Beulich

Quoted 7 lines
Quoted 1 line
Quoted 1 line · On 12.06.18 at 16:10, <[email protected]> wrote:
Since qemu-2.10 it is required to unplug emulated devices again after a live migration. If this is not done, qemu's block-backend driver will be unable to open the backing disk image because it is still busy by qemu's IDE driver. As a result the domUs block-frontend driver will be unable to access the disks, and the domU has to be destroyed. libxl is unable to detect the situation.
So there's no plan to fix this on the qemu side?

Quoted 7 lines

Quoted 4 lines
Apply the same workaround for this qemu bug that was done already years ago in linux.git with commit 512b109ec962 ("xen: unplug the emulated devices at resume time") to make sure xenlinux based domUs can be migrated to unfixed hosts.
Hmm, that change has no explanation whatsoever of why this was necessary already back at that time. Stefano?

It was too long ago, I don't remember exactly, but my guess is that the presence of the emulated devices at resume time was causing problems to Linux. Given that Linux HVM was supposed to be able to run on older versions of Xen, the workaround was put in Linux rather than Xen. (I wonder if the Windows PV drivers have also something similar.)

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Jan Beulich

2018-06-13 06:27 UTC #4 lore raw

Quoted 17 lines

Quoted 1 line
Quoted 1 line · On 12.06.18 at 21:39, <[email protected]> wrote:
Quoted 15 lines · Jan Beulich
Quoted 7 lines
Quoted 1 line
Quoted 1 line · On 12.06.18 at 16:10, <[email protected]> wrote:
Since qemu-2.10 it is required to unplug emulated devices again after a live migration. If this is not done, qemu's block-backend driver will be unable to open the backing disk image because it is still busy by qemu's IDE driver. As a result the domUs block-frontend driver will be unable to access the disks, and the domU has to be destroyed. libxl is unable to detect the situation.
So there's no plan to fix this on the qemu side?
Quoted 4 lines
Apply the same workaround for this qemu bug that was done already years ago in linux.git with commit 512b109ec962 ("xen: unplug the emulated devices at resume time") to make sure xenlinux based domUs can be migrated to unfixed hosts.

Quoted 8 lines

Quoted 2 lines
Hmm, that change has no explanation whatsoever of why this was necessary already back at that time. Stefano?
It was too long ago, I don't remember exactly, but my guess is that the presence of the emulated devices at resume time was causing problems to Linux. Given that Linux HVM was supposed to be able to run on older versions of Xen, the workaround was put in Linux rather than Xen. (I wonder if the Windows PV drivers have also something similar.)

Paul?

Thanks, Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

Paul Durrant

2018-06-13 08:16 UTC #5 lore raw

Quoted 29 lines

-----Original Message-----
From: Jan Beulich [mailto:[email protected]]
Sent: 13 June 2018 07:27
To: Paul Durrant <[email protected]>
Cc: Olaf Hering <[email protected]>; Stefano Stabellini <[email protected]>; [email protected]
Subject: Re: [PATCH v1] unmodified_drivers: unplug the emulated devices at resume time
Quoted 20 lines
Quoted 1 line
Quoted 1 line · On 12.06.18 at 21:39, <[email protected]> wrote:
Quoted 17 lines · Jan Beulich
Quoted 7 lines
Quoted 1 line
Quoted 1 line · On 12.06.18 at 16:10, <[email protected]> wrote:
Since qemu-2.10 it is required to unplug emulated devices again after a live migration. If this is not done, qemu's block-backend driver will be unable to open the backing disk image because it is still busy by qemu's IDE driver. As a result the domUs block-frontend driver will be unable to access the disks, and the domU has to be destroyed. libxl is unable to detect the situation.
So there's no plan to fix this on the qemu side?
Quoted 4 lines
Apply the same workaround for this qemu bug that was done already years ago in linux.git with commit 512b109ec962 ("xen: unplug the emulated devices at resume time") to make sure xenlinux based domUs can be migrated to unfixed hosts.
Hmm, that change has no explanation whatsoever of why this was necessary already back at that time. Stefano?

Quoted 7 lines

Quoted 5 lines
It was too long ago, I don't remember exactly, but my guess is that the presence of the emulated devices at resume time was causing problems to Linux. Given that Linux HVM was supposed to be able to run on older versions of Xen, the workaround was put in Linux rather than Xen. (I wonder if the Windows PV drivers have also something similar.)
Paul?

Yes, they do. Emulated devices are unplugged on resume, just after the hypercall table is updated. See:

http://xenbits.xen.org/gitweb/?p=pvdrivers/win/xenbus.git;a=blob;f=src/xenbus/suspend.c;hb=HEAD#l219

AFAIK this has always done (dating back to the ancient Citrix drivers). I suspect you'll find the same in GPLPV.

Paul

Quoted 3 lines

Thanks, Jan

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel