Skip to contentSkip to conversation

xen-blkback: support live update

PATCH Superseded Paul Durrant 3 participants 38 messages Dec 5, 2019 – Dec 10, 2019

Subsystems (3)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

cover xen-blkback: support live update

Patch #1 is clean-up for an apparent mis-feature.

Paul Durrant (4):
xenbus: move xenbus_dev_shutdown() into frontend code...
xenbus: limit when state is forced to closed xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH
xen-blkback: support dynamic unbind/bind

 drivers/block/xen-blkback/xenbus.c         | 12 ++++----
 drivers/xen/xenbus/xenbus.h                |  2 --
 drivers/xen/xenbus/xenbus_probe.c          | 34 ++++++----------------
 drivers/xen/xenbus/xenbus_probe_backend.c  |  1 -
 drivers/xen/xenbus/xenbus_probe_frontend.c | 24 ++++++++++++++-
 include/xen/interface/io/ring.h            |  4 +--
 6 files changed, 40 insertions(+), 37 deletions(-)

---
Cc: Boris Ostrovsky <[email protected]>
Cc: Jens Axboe <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
Cc: Stefano Stabellini <[email protected]>

Signature · 1 line
2.20.1
#1 lore raw

PATCH 1/4 xenbus: move xenbus_dev_shutdown() into frontend code...

...and make it static

xenbus_dev_shutdown() is seemingly intended to cause clean shutdown of PV frontends when a guest is rebooted. Indeed the function waits for a conpletion which is only set by a call to xenbus_frontend_closed().

This patch removes the shutdown() method from backends and moves xenbus_dev_shutdown() from xenbus_probe.c into xenbus_probe_frontend.c, renaming it appropriately and making it static.

---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Stefano Stabellini <[email protected]>

---
 drivers/xen/xenbus/xenbus.h                |  2 --
 drivers/xen/xenbus/xenbus_probe.c          | 23 ---------------------
 drivers/xen/xenbus/xenbus_probe_backend.c  |  1 -
 drivers/xen/xenbus/xenbus_probe_frontend.c | 24 +++++++++++++++++++++-
 4 files changed, 23 insertions(+), 27 deletions(-)
drivers/xen/xenbus/xenbus.h +0 -2
index d75a2385b37c..5f5b8a7d5b80 100644
@@ -116,8 +116,6 @@ int xenbus_probe_devices(struct xen_bus_type *bus);
 
 void xenbus_dev_changed(const char *node, struct xen_bus_type *bus);
 
-void xenbus_dev_shutdown(struct device *_dev);
-
 int xenbus_dev_suspend(struct device *dev);
 int xenbus_dev_resume(struct device *dev);
 int xenbus_dev_cancel(struct device *dev);
drivers/xen/xenbus/xenbus_probe.c +0 -23
index 4461f4583476..a10311c348b9 100644
@@ -281,29 +281,6 @@ int xenbus_dev_remove(struct device *_dev)
 }
 EXPORT_SYMBOL_GPL(xenbus_dev_remove);
 
-void xenbus_dev_shutdown(struct device *_dev)
-{
-	struct xenbus_device *dev = to_xenbus_device(_dev);
-	unsigned long timeout = 5*HZ;
-
-	DPRINTK("%s", dev->nodename);
-
-	get_device(&dev->dev);
-	if (dev->state != XenbusStateConnected) {
-		pr_info("%s: %s: %s != Connected, skipping\n",
-			__func__, dev->nodename, xenbus_strstate(dev->state));
-		goto out;
-	}
-	xenbus_switch_state(dev, XenbusStateClosing);
-	timeout = wait_for_completion_timeout(&dev->down, timeout);
-	if (!timeout)
-		pr_info("%s: %s timeout closing device\n",
-			__func__, dev->nodename);
- out:
-	put_device(&dev->dev);
-}
-EXPORT_SYMBOL_GPL(xenbus_dev_shutdown);
-
 int xenbus_register_driver_common(struct xenbus_driver *drv,
 				  struct xen_bus_type *bus,
 				  struct module *owner, const char *mod_name)
drivers/xen/xenbus/xenbus_probe_backend.c +0 -1
index b0bed4faf44c..14876faff3b0 100644
@@ -198,7 +198,6 @@ static struct xen_bus_type xenbus_backend = {
 		.uevent		= xenbus_uevent_backend,
 		.probe		= xenbus_dev_probe,
 		.remove		= xenbus_dev_remove,
-		.shutdown	= xenbus_dev_shutdown,
 		.dev_groups	= xenbus_dev_groups,
 	},
 };
drivers/xen/xenbus/xenbus_probe_frontend.c +23 -1
index a7d90a719cea..8a1650bbe18f 100644
@@ -126,6 +126,28 @@ static int xenbus_frontend_dev_probe(struct device *dev)
 	return xenbus_dev_probe(dev);
 }
 
+static void xenbus_frontend_dev_shutdown(struct device *_dev)
+{
+	struct xenbus_device *dev = to_xenbus_device(_dev);
+	unsigned long timeout = 5*HZ;
+
+	DPRINTK("%s", dev->nodename);
+
+	get_device(&dev->dev);
+	if (dev->state != XenbusStateConnected) {
+		pr_info("%s: %s: %s != Connected, skipping\n",
+			__func__, dev->nodename, xenbus_strstate(dev->state));
+		goto out;
+	}
+	xenbus_switch_state(dev, XenbusStateClosing);
+	timeout = wait_for_completion_timeout(&dev->down, timeout);
+	if (!timeout)
+		pr_info("%s: %s timeout closing device\n",
+			__func__, dev->nodename);
+ out:
+	put_device(&dev->dev);
+}
+
 static const struct dev_pm_ops xenbus_pm_ops = {
 	.suspend	= xenbus_dev_suspend,
 	.resume		= xenbus_frontend_dev_resume,
@@ -146,7 +168,7 @@ static struct xen_bus_type xenbus_frontend = {
 		.uevent		= xenbus_uevent_frontend,
 		.probe		= xenbus_frontend_dev_probe,
 		.remove		= xenbus_dev_remove,
-		.shutdown	= xenbus_dev_shutdown,
+		.shutdown	= xenbus_frontend_dev_shutdown,
 		.dev_groups	= xenbus_dev_groups,
 
 		.pm		= &xenbus_pm_ops,
Signature · 1 line
2.20.1
maintainer
#2 lore raw

Re: [PATCH 1/4] xenbus: move xenbus_dev_shutdown() into frontend code...

Quoted 9 lines · Paul Durrant

...and make it static

xenbus_dev_shutdown() is seemingly intended to cause clean shutdown of PV frontends when a guest is rebooted. Indeed the function waits for a conpletion which is only set by a call to xenbus_frontend_closed().

This patch removes the shutdown() method from backends and moves xenbus_dev_shutdown() from xenbus_probe.c into xenbus_probe_frontend.c, renaming it appropriately and making it static.

Is this a good move considering driver domains?

At least I'd expect the commit message addressing the expected behavior with rebooting a driver domain and why this patch isn't making things worse.

Juergen

#3 lore raw

RE: [PATCH 1/4] xenbus: move xenbus_dev_shutdown() into frontend code...

Quoted 15 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:34
To: Durrant, Paul <[email protected]>; [email protected]; [email protected]
Cc: Boris Ostrovsky <[email protected]>; Stefano Stabellini <[email protected]>
Subject: Re: [PATCH 1/4] xenbus: move xenbus_dev_shutdown() into frontend code...

Quoted 3 lines · Paul Durrant

...and make it static

xenbus_dev_shutdown() is seemingly intended to cause clean shutdown of

PV

Quoted 8 lines
Quoted 6 lines

frontends when a guest is rebooted. Indeed the function waits for a conpletion which is only set by a call to xenbus_frontend_closed().

This patch removes the shutdown() method from backends and moves xenbus_dev_shutdown() from xenbus_probe.c into xenbus_probe_frontend.c, renaming it appropriately and making it static.

Is this a good move considering driver domains?

I don't think it can have ever worked properly for driver domains, and with the rest of the patches a backend should be able go away and return unannounced (as long as the domain id is kept... for which patches need to be upstreamed into Xen).

Quoted 5 lines

At least I'd expect the commit message addressing the expected behavior with rebooting a driver domain and why this patch isn't making things worse.

For a clean reboot I'd expect the toolstack to shut down the protocol before rebooting the driver domain, so the backend shutdown method is moot. And I don't believe re-startable driver domains were something that ever made it into support (because of the non-persistent domid problem). I can add something to the commit comment to that effect if you'd like.

Paul

maintainer Reviewed-by
#4 lore raw

Re: [PATCH 1/4] xenbus: move xenbus_dev_shutdown() into frontend code...

Quoted 27 lines · Paul
Quoted 15 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:34
To: Durrant, Paul <[email protected]>; [email protected]; [email protected]
Cc: Boris Ostrovsky <[email protected]>; Stefano Stabellini <[email protected]>
Subject: Re: [PATCH 1/4] xenbus: move xenbus_dev_shutdown() into frontend code...

Quoted 3 lines · Paul Durrant

...and make it static

xenbus_dev_shutdown() is seemingly intended to cause clean shutdown of

PV

Quoted 8 lines
Quoted 6 lines

frontends when a guest is rebooted. Indeed the function waits for a conpletion which is only set by a call to xenbus_frontend_closed().

This patch removes the shutdown() method from backends and moves xenbus_dev_shutdown() from xenbus_probe.c into xenbus_probe_frontend.c, renaming it appropriately and making it static.

Is this a good move considering driver domains?

I don't think it can have ever worked properly for driver domains, and with the rest of the patches a backend should be able go away and return unannounced (as long as the domain id is kept... for which patches need to be upstreamed into Xen).

Quoted 1 line
Quoted 6 lines
Quoted 4 lines

At least I'd expect the commit message addressing the expected behavior with rebooting a driver domain and why this patch isn't making things worse.

For a clean reboot I'd expect the toolstack to shut down the protocol before rebooting the driver domain, so the backend shutdown method is moot. And I don't believe re-startable driver domains were something that ever made it into support (because of the non-persistent domid problem). I can add something to the commit comment to that effect if you'd like.

Yes, please do so.

With this you can add my:

Juergen

#5 lore raw

PATCH 2/4 xenbus: limit when state is forced to closed

In reply to Paul Durrant

Only force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in xenstore has been set to closing prior to device removal.

---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Stefano Stabellini <[email protected]>

---
 drivers/xen/xenbus/xenbus_probe.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
drivers/xen/xenbus/xenbus_probe.c +9 -2
index a10311c348b9..c54a53da0106 100644
@@ -255,7 +255,6 @@ int xenbus_dev_probe(struct device *_dev)
 	module_put(drv->driver.owner);
 fail:
 	xenbus_dev_error(dev, err, "xenbus_dev_probe on %s", dev->nodename);
-	xenbus_switch_state(dev, XenbusStateClosed);
 	return err;
 }
 EXPORT_SYMBOL_GPL(xenbus_dev_probe);
@@ -264,6 +263,7 @@ int xenbus_dev_remove(struct device *_dev)
 {
 	struct xenbus_device *dev = to_xenbus_device(_dev);
 	struct xenbus_driver *drv = to_xenbus_driver(_dev->driver);
+	enum xenbus_state state;
 
 	DPRINTK("%s", dev->nodename);
 
@@ -276,7 +276,14 @@ int xenbus_dev_remove(struct device *_dev)
 
 	free_otherend_details(dev);
 
-	xenbus_switch_state(dev, XenbusStateClosed);
+	/*
+	 * If the toolstack had force the device state to closing then set
+	 * the state to closed now to allow it to be cleaned up.
+	 */
+	state = xenbus_read_driver_state(dev->nodename);
+	if (state == XenbusStateClosing)
+		xenbus_switch_state(dev, XenbusStateClosed);
+
 	return 0;
 }
 EXPORT_SYMBOL_GPL(xenbus_dev_remove);
Signature · 1 line
2.20.1
#6 lore raw

Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 3 lines · Paul Durrant

Only force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in xenstore has been set to closing prior to device removal.

I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.

Can you explain what's the benefit of leaving a device without a driver in such unknown state?

Thanks, Roger.

maintainer
#7 lore raw

Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 11 lines · Roger Pau Monné
Quoted 3 lines · Paul Durrant

Only force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in xenstore has been set to closing prior to device removal.

I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.

Can you explain what's the benefit of leaving a device without a driver in such unknown state?

And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having been set before?

Juergen

#8 lore raw

RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 22 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 3 lines · Roger Pau Monné
Quoted 2 lines · Paul Durrant

Only force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in

xenstore

Quoted 6 lines
Quoted 1 line

has been set to closing prior to device removal.

I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.

Quoted 6 lines
Quoted 2 lines

Can you explain what's the benefit of leaving a device without a driver in such unknown state?

And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having been set before?

Blkfront doesn't seem to mind and I believe the Windows PV drivers cope, but I don't really understand the comment since this patch is actually removing a case where the backend transitions directly to closed.

Paul

Quoted 3 lines

Juergen

maintainer
#9 lore raw

Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 22 lines · Paul
Quoted 22 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 3 lines · Roger Pau Monné
Quoted 2 lines · Paul Durrant

Only force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in

xenstore

Quoted 6 lines
Quoted 1 line

has been set to closing prior to device removal.

I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

Can you explain what's the benefit of leaving a device without a driver in such unknown state?

And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having been set before?

Blkfront doesn't seem to mind and I believe the Windows PV drivers cope, but I don't really understand the comment since this patch is actually removing a case where the backend transitions directly to closed.

I'm not speaking of blkfront/blkback only, but of net, tpm, scsi, pvcall etc. frontends/backends. After all you are modifying a function common to all PV driver pairs.

You are removing a state switc to "closed" in case the state was _not_ "closing" before. So any PV driver reacting to "closed" of the other end in case the previous state might not have been "closing" before is at risk to misbehave with your patch.

Juergen

#10 lore raw

RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 47 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 12:09
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 6 lines · Paul
Quoted 6 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];

Stefano

Quoted 3 lines
Quoted 3 lines

Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced

to

Quoted 13 lines
Quoted 13 lines

closed

Quoted 3 lines · Roger Pau Monné
Quoted 2 lines · Paul Durrant

Only force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in

xenstore

Quoted 6 lines
Quoted 1 line

has been set to closing prior to device removal.

I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

Can you explain what's the benefit of leaving a device without a driver in such unknown state?

And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having been set before?

Blkfront doesn't seem to mind and I believe the Windows PV drivers cope,

but I don't really understand the comment since this patch is actually removing a case where the backend transitions directly to closed.

Quoted 6 lines

I'm not speaking of blkfront/blkback only, but of net, tpm, scsi, pvcall etc. frontends/backends. After all you are modifying a function common to all PV driver pairs.

You are removing a state switc to "closed" in case the state was _not_ "closing" before.

Yes, which AFAIK is against the intention of the generic PV protocol such that it ever existed anyway.

Quoted 3 lines

So any PV driver reacting to "closed" of the other end
in case the previous state might not have been "closing" before is at risk to misbehave with your patch.

Well, they will see nothing now. If the state was not closing, it gets left alone, so the frontend shouldn't do anything. The only risk that I can see is that some frontend/backend pair needed a direct 4 -> 6 transition to support 'unbind' before but AFAIK nothing has ever supported that, and blk and net crash'n'burn if you try that on upstream as it stands. A clean unplug would always set state to 5 first, since that's part of the unplug protocol.

Paul

Quoted 2 lines

Juergen

maintainer
#11 lore raw

Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 47 lines · Paul
Quoted 47 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 12:09
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 6 lines · Paul
Quoted 6 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];

Stefano

Quoted 3 lines
Quoted 3 lines

Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced

to

Quoted 13 lines
Quoted 13 lines

closed

Quoted 3 lines · Roger Pau Monné
Quoted 2 lines · Paul Durrant

Only force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in

xenstore

Quoted 6 lines
Quoted 1 line

has been set to closing prior to device removal.

I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

Can you explain what's the benefit of leaving a device without a driver in such unknown state?

And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having been set before?

Blkfront doesn't seem to mind and I believe the Windows PV drivers cope,

but I don't really understand the comment since this patch is actually removing a case where the backend transitions directly to closed.

Quoted 8 lines
Quoted 6 lines

I'm not speaking of blkfront/blkback only, but of net, tpm, scsi, pvcall etc. frontends/backends. After all you are modifying a function common to all PV driver pairs.

You are removing a state switc to "closed" in case the state was _not_ "closing" before.

Yes, which AFAIK is against the intention of the generic PV protocol such that it ever existed anyway.

While this might be the case we should _not_ break any guests running now. So this kind of reasoning is dangerous.

Quoted 6 lines
Quoted 4 lines

So any PV driver reacting to "closed" of the other end
in case the previous state might not have been "closing" before is at risk to misbehave with your patch.

Well, they will see nothing now. If the state was not closing, it gets left alone, so the frontend shouldn't do anything. The only risk that I can see is that some frontend/backend pair needed a direct 4 -> 6 transition to support 'unbind' before but AFAIK nothing has ever supported that, and blk and net crash'n'burn if you try that on upstream as it stands. A clean unplug would always set state to 5 first, since that's part of the unplug protocol.

That was my question: are you sure all current and previous guest frontends and backends are handling unplug this way?

Not "should handle", but "do handle".

Juergen

#12 lore raw

RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 92 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 13:39
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 6 lines · Paul
Quoted 6 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 12:09
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];

Stefano

Quoted 3 lines
Quoted 3 lines

Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced

to

Quoted 9 lines
Quoted 9 lines

closed

Quoted 6 lines · Paul
Quoted 6 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];

Quoted 4 lines
Quoted 4 lines

Stefano

Quoted 3 lines
Quoted 3 lines

Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is

forced

Quoted 6 lines
Quoted 6 lines

to

Quoted 5 lines
Quoted 5 lines

closed

Quoted 2 lines · Roger Pau Monné
Quoted 1 line · Paul Durrant

Only force state to closed in the case when the toolstack may need

to

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line

clean up. This can be detected by checking whether the state in

xenstore

Quoted 3 lines
Quoted 1 line

has been set to closing prior to device removal.

I'm not sure I see the point of this, I would expect that a failure

to

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 9 lines
Quoted 5 lines

probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.

Can you explain what's the benefit of leaving a device without a driver in such unknown state?

And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having been set before?

Blkfront doesn't seem to mind and I believe the Windows PV drivers

cope,

Quoted 4 lines
Quoted 4 lines

but I don't really understand the comment since this patch is actually removing a case where the backend transitions directly to closed.

I'm not speaking of blkfront/blkback only, but of net, tpm, scsi,

pvcall

Quoted 7 lines
Quoted 5 lines

etc. frontends/backends. After all you are modifying a function common to all PV driver pairs.

You are removing a state switc to "closed" in case the state was _not_ "closing" before.

Yes, which AFAIK is against the intention of the generic PV protocol

such that it ever existed anyway.

While this might be the case we should _not_ break any guests running now. So this kind of reasoning is dangerous.

Quoted 6 lines
Quoted 4 lines

So any PV driver reacting to "closed" of the other end
in case the previous state might not have been "closing" before is at risk to misbehave with your patch.

Well, they will see nothing now. If the state was not closing, it gets

left alone, so the frontend shouldn't do anything. The only risk that I can see is that some frontend/backend pair needed a direct 4 -> 6 transition to support 'unbind' before but AFAIK nothing has ever supported that, and blk and net crash'n'burn if you try that on upstream as it stands. A clean unplug would always set state to 5 first, since that's part of the unplug protocol.

Quoted 4 lines

That was my question: are you sure all current and previous guest frontends and backends are handling unplug this way?

Not "should handle", but "do handle".

That depends on the toolstack. IIUC the only 'supported' toolstack is xl/libxl, which will set 'state' to 5 and 'online' to 0 to initiate an unplug.

Paul

Quoted 3 lines

Juergen

maintainer
#13 lore raw

Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 92 lines · Paul
Quoted 92 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 13:39
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 6 lines · Paul
Quoted 6 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 12:09
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];

Stefano

Quoted 3 lines
Quoted 3 lines

Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced

to

Quoted 9 lines
Quoted 9 lines

closed

Quoted 6 lines · Paul
Quoted 6 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];

Quoted 4 lines
Quoted 4 lines

Stefano

Quoted 3 lines
Quoted 3 lines

Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is

forced

Quoted 6 lines
Quoted 6 lines

to

Quoted 5 lines
Quoted 5 lines

closed

Quoted 2 lines · Roger Pau Monné
Quoted 1 line · Paul Durrant

Only force state to closed in the case when the toolstack may need

to

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line

clean up. This can be detected by checking whether the state in

xenstore

Quoted 3 lines
Quoted 1 line

has been set to closing prior to device removal.

I'm not sure I see the point of this, I would expect that a failure

to

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 9 lines
Quoted 5 lines

probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.

Can you explain what's the benefit of leaving a device without a driver in such unknown state?

And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having been set before?

Blkfront doesn't seem to mind and I believe the Windows PV drivers

cope,

Quoted 4 lines
Quoted 4 lines

but I don't really understand the comment since this patch is actually removing a case where the backend transitions directly to closed.

I'm not speaking of blkfront/blkback only, but of net, tpm, scsi,

pvcall

Quoted 7 lines
Quoted 5 lines

etc. frontends/backends. After all you are modifying a function common to all PV driver pairs.

You are removing a state switc to "closed" in case the state was _not_ "closing" before.

Yes, which AFAIK is against the intention of the generic PV protocol

such that it ever existed anyway.

While this might be the case we should _not_ break any guests running now. So this kind of reasoning is dangerous.

Quoted 6 lines
Quoted 4 lines

So any PV driver reacting to "closed" of the other end
in case the previous state might not have been "closing" before is at risk to misbehave with your patch.

Well, they will see nothing now. If the state was not closing, it gets

left alone, so the frontend shouldn't do anything. The only risk that I can see is that some frontend/backend pair needed a direct 4 -> 6 transition to support 'unbind' before but AFAIK nothing has ever supported that, and blk and net crash'n'burn if you try that on upstream as it stands. A clean unplug would always set state to 5 first, since that's part of the unplug protocol.

Quoted 6 lines
Quoted 4 lines

That was my question: are you sure all current and previous guest frontends and backends are handling unplug this way?

Not "should handle", but "do handle".

That depends on the toolstack. IIUC the only 'supported' toolstack is xl/libxl, which will set 'state' to 5 and 'online' to 0 to initiate an unplug.

I guess libvirt/libxl is doing the same?

At least at SUSE we still have some customers running xend based Xen installations with recent Linux or Windows guests.

Juergen

#14 lore raw

RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 117 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 14:10
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 6 lines · Paul
Quoted 6 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 13:39
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];

Stefano

Quoted 3 lines
Quoted 3 lines

Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced

to

Quoted 9 lines
Quoted 9 lines

closed

Quoted 6 lines · Paul
Quoted 6 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 12:09
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];

Quoted 4 lines
Quoted 4 lines

Stefano

Quoted 3 lines
Quoted 3 lines

Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is

forced

Quoted 14 lines
Quoted 14 lines

to

Quoted 13 lines
Quoted 13 lines

closed

Quoted 6 lines · Paul
Quoted 6 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];

Stefano

Quoted 3 lines
Quoted 3 lines

Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is

Quoted 7 lines
Quoted 7 lines

forced

Quoted 6 lines
Quoted 6 lines

to

Quoted 5 lines
Quoted 5 lines

closed

Quoted 2 lines · Roger Pau Monné
Quoted 1 line · Paul Durrant

Only force state to closed in the case when the toolstack may

need

Quoted 6 lines
Quoted 6 lines

to

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line

clean up. This can be detected by checking whether the state in

xenstore

Quoted 3 lines
Quoted 1 line

has been set to closing prior to device removal.

I'm not sure I see the point of this, I would expect that a

failure

Quoted 9 lines
Quoted 9 lines

to

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.

Can you explain what's the benefit of leaving a device without a driver in such unknown state?

And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having

been

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

set before?

Blkfront doesn't seem to mind and I believe the Windows PV drivers

cope,

Quoted 1 line
Quoted 1 line

but I don't really understand the comment since this patch is

actually

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines

removing a case where the backend transitions directly to closed.

I'm not speaking of blkfront/blkback only, but of net, tpm, scsi,

pvcall

Quoted 1 line
Quoted 1 line

etc. frontends/backends. After all you are modifying a function

common

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

to all PV driver pairs.

You are removing a state switc to "closed" in case the state was

_not_

Quoted 9 lines
Quoted 9 lines
Quoted 3 lines
Quoted 1 line

"closing" before.

Yes, which AFAIK is against the intention of the generic PV protocol

such that it ever existed anyway.

While this might be the case we should _not_ break any guests running now. So this kind of reasoning is dangerous.

Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

So any PV driver reacting to "closed" of the other end
in case the previous state might not have been "closing" before is at risk to misbehave with your patch.

Well, they will see nothing now. If the state was not closing, it gets

left alone, so the frontend shouldn't do anything. The only risk that I can see is that some frontend/backend pair needed a direct 4 -> 6 transition to support 'unbind' before but AFAIK nothing has ever

supported

Quoted 7 lines
Quoted 7 lines

that, and blk and net crash'n'burn if you try that on upstream as it stands. A clean unplug would always set state to 5 first, since that's part of the unplug protocol.

That was my question: are you sure all current and previous guest frontends and backends are handling unplug this way?

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Not "should handle", but "do handle".

That depends on the toolstack. IIUC the only 'supported' toolstack is

xl/libxl, which will set 'state' to 5 and 'online' to 0 to initiate an unplug.

I guess libvirt/libxl is doing the same?

The unplug mechansism is all in libxl AFAICT, so it should be identical.

Quoted 3 lines

At least at SUSE we still have some customers running xend based Xen installations with recent Linux or Windows guests.

Is that something the upstream code can/should support though? I'd be surprised if xend is actually doing anything different to libxl since I've been coding the Windows PV drivers to trigger off the combined closing/online transition for as long as I can remember.

Paul

maintainer
#15 lore raw

Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 126 lines · Paul
Quoted 117 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 14:10
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 6 lines · Paul
Quoted 6 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 13:39
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];

Stefano

Quoted 3 lines
Quoted 3 lines

Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced

to

Quoted 9 lines
Quoted 9 lines

closed

Quoted 6 lines · Paul
Quoted 6 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 12:09
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];

Quoted 4 lines
Quoted 4 lines

Stefano

Quoted 3 lines
Quoted 3 lines

Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is

forced

Quoted 14 lines
Quoted 14 lines

to

Quoted 13 lines
Quoted 13 lines

closed

Quoted 6 lines · Paul
Quoted 6 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];

Stefano

Quoted 3 lines
Quoted 3 lines

Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is

Quoted 7 lines
Quoted 7 lines

forced

Quoted 6 lines
Quoted 6 lines

to

Quoted 5 lines
Quoted 5 lines

closed

Quoted 2 lines · Roger Pau Monné
Quoted 1 line · Paul Durrant

Only force state to closed in the case when the toolstack may

need

Quoted 6 lines
Quoted 6 lines

to

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line

clean up. This can be detected by checking whether the state in

xenstore

Quoted 3 lines
Quoted 1 line

has been set to closing prior to device removal.

I'm not sure I see the point of this, I would expect that a

failure

Quoted 9 lines
Quoted 9 lines

to

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines

probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.

Can you explain what's the benefit of leaving a device without a driver in such unknown state?

And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having

been

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

set before?

Blkfront doesn't seem to mind and I believe the Windows PV drivers

cope,

Quoted 1 line
Quoted 1 line

but I don't really understand the comment since this patch is

actually

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines

removing a case where the backend transitions directly to closed.

I'm not speaking of blkfront/blkback only, but of net, tpm, scsi,

pvcall

Quoted 1 line
Quoted 1 line

etc. frontends/backends. After all you are modifying a function

common

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

to all PV driver pairs.

You are removing a state switc to "closed" in case the state was

_not_

Quoted 9 lines
Quoted 9 lines
Quoted 3 lines
Quoted 1 line

"closing" before.

Yes, which AFAIK is against the intention of the generic PV protocol

such that it ever existed anyway.

While this might be the case we should _not_ break any guests running now. So this kind of reasoning is dangerous.

Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

So any PV driver reacting to "closed" of the other end
in case the previous state might not have been "closing" before is at risk to misbehave with your patch.

Well, they will see nothing now. If the state was not closing, it gets

left alone, so the frontend shouldn't do anything. The only risk that I can see is that some frontend/backend pair needed a direct 4 -> 6 transition to support 'unbind' before but AFAIK nothing has ever

supported

Quoted 7 lines
Quoted 7 lines

that, and blk and net crash'n'burn if you try that on upstream as it stands. A clean unplug would always set state to 5 first, since that's part of the unplug protocol.

That was my question: are you sure all current and previous guest frontends and backends are handling unplug this way?

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Not "should handle", but "do handle".

That depends on the toolstack. IIUC the only 'supported' toolstack is

xl/libxl, which will set 'state' to 5 and 'online' to 0 to initiate an unplug.

I guess libvirt/libxl is doing the same?

Quoted 7 lines

The unplug mechansism is all in libxl AFAICT, so it should be identical.

Quoted 3 lines

At least at SUSE we still have some customers running xend based Xen installations with recent Linux or Windows guests.

Is that something the upstream code can/should support though? I'd be surprised if xend is actually doing anything different to libxl since I've been coding the Windows PV drivers to trigger off the combined closing/online transition for as long as I can remember.

I'd rather not have to carry a private patch for new Linux kernel to be able to run on those hosts.

AFAIK you at Amazon have some quite old Xen installations, too. How are you handling that (assuming the customer is updating the kernel to a recent version in his guest)?

Juergen

#16 lore raw

RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 157 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 14:41
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 6 lines · Paul
Quoted 6 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 14:10
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];

Stefano

Quoted 3 lines
Quoted 3 lines

Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced

to

Quoted 9 lines
Quoted 9 lines

closed

Quoted 6 lines · Paul
Quoted 6 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 13:39
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];

Quoted 4 lines
Quoted 4 lines

Stefano

Quoted 3 lines
Quoted 3 lines

Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is

forced

Quoted 38 lines
Quoted 38 lines

to

Quoted 9 lines
Quoted 9 lines

closed

Quoted 6 lines · Paul
Quoted 6 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 12:09
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];

Quoted 4 lines
Quoted 4 lines

Stefano

Quoted 3 lines
Quoted 3 lines

Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is

forced

Quoted 14 lines
Quoted 14 lines

to

Quoted 13 lines
Quoted 13 lines

closed

Quoted 6 lines · Paul
Quoted 6 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];

Stefano

Quoted 3 lines
Quoted 3 lines

Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is

Quoted 7 lines
Quoted 7 lines

forced

Quoted 6 lines
Quoted 6 lines

to

Quoted 5 lines
Quoted 5 lines

closed

Quoted 2 lines · Roger Pau Monné
Quoted 1 line · Paul Durrant

Only force state to closed in the case when the toolstack may

need

Quoted 1 line
Quoted 1 line

to

Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line

clean up. This can be detected by checking whether the state in

xenstore

Quoted 3 lines
Quoted 1 line

has been set to closing prior to device removal.

I'm not sure I see the point of this, I would expect that a

failure

Quoted 2 lines
Quoted 2 lines

to

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

probe or the removal of the device would leave the xenbus state

as

Quoted 27 lines
Quoted 27 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines

closed, which is consistent with the actual driver state.

Can you explain what's the benefit of leaving a device without a driver in such unknown state?

And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having

been

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

set before?

Blkfront doesn't seem to mind and I believe the Windows PV drivers

cope,

Quoted 1 line
Quoted 1 line

but I don't really understand the comment since this patch is

actually

Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines

removing a case where the backend transitions directly to closed.

I'm not speaking of blkfront/blkback only, but of net, tpm, scsi,

pvcall

Quoted 1 line
Quoted 1 line

etc. frontends/backends. After all you are modifying a function

common

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

to all PV driver pairs.

You are removing a state switc to "closed" in case the state was

_not_

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

"closing" before.

Yes, which AFAIK is against the intention of the generic PV protocol

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

such that it ever existed anyway.

While this might be the case we should _not_ break any guests running now. So this kind of reasoning is dangerous.

Quoted 3 lines
Quoted 3 lines

So any PV driver reacting to "closed" of the other end in case the previous state might not have been "closing" before is

at

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line

risk to misbehave with your patch.

Well, they will see nothing now. If the state was not closing, it

gets

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

left alone, so the frontend shouldn't do anything. The only risk that

I

Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines

can see is that some frontend/backend pair needed a direct 4 -> 6 transition to support 'unbind' before but AFAIK nothing has ever

supported

Quoted 2 lines
Quoted 2 lines

that, and blk and net crash'n'burn if you try that on upstream as it stands. A clean unplug would always set state to 5 first, since

that's

Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

part of the unplug protocol.

That was my question: are you sure all current and previous guest frontends and backends are handling unplug this way?

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Not "should handle", but "do handle".

That depends on the toolstack. IIUC the only 'supported' toolstack is

xl/libxl, which will set 'state' to 5 and 'online' to 0 to initiate an unplug.

I guess libvirt/libxl is doing the same?

Quoted 7 lines

The unplug mechansism is all in libxl AFAICT, so it should be identical.

Quoted 3 lines

At least at SUSE we still have some customers running xend based Xen installations with recent Linux or Windows guests.

Is that something the upstream code can/should support though? I'd be

surprised if xend is actually doing anything different to libxl since I've been coding the Windows PV drivers to trigger off the combined closing/online transition for as long as I can remember.

Quoted 6 lines

I'd rather not have to carry a private patch for new Linux kernel to be able to run on those hosts.

AFAIK you at Amazon have some quite old Xen installations, too. How are you handling that (assuming the customer is updating the kernel to a recent version in his guest)?

I'm not aware of any problems running with xend (but I'm not I the loop on everything). I think it is still a reasonably safe assumption that xend initiated unplug cleanly and doesn't rely on a 4 -> 6 transition.

Paul

Quoted 3 lines

Juergen

#17 lore raw

RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

In reply to Roger Pau Monné
Quoted 15 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 11:39
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Juergen Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 3 lines · Paul Durrant

Only force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in xenstore has been set to closing prior to device removal.

Quoted 7 lines

I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.

Can you explain what's the benefit of leaving a device without a driver in such unknown state?

If probe fails then I think it should leave the state alone. If the state is moved to closed then basically you just killed that connection to the guest (as the frontend will normally close down when it sees this change) so, if the probe failure was due to a bug in blkback or, e.g., a transient resource issue then it's game over as far as that guest goes. The ultimate goal here is PV backend re-load that is completely transparent to the guest. Modifying anything in xenstore compromises that so we need to be careful.

Paul

#18 lore raw

Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 23 lines · Paul
Quoted 15 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 11:39
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Juergen Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 3 lines · Paul Durrant

Only force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in xenstore has been set to closing prior to device removal.

Quoted 7 lines

I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.

Can you explain what's the benefit of leaving a device without a driver in such unknown state?

Quoted 6 lines

If probe fails then I think it should leave the state alone. If the state is moved to closed then basically you just killed that connection to the guest (as the frontend will normally close down when it sees this change) so, if the probe failure was due to a bug in blkback or, e.g., a transient resource issue then it's game over as far as that guest goes.

But the connection can be restarted by switching the backend to the init state again.

Quoted 1 line

The ultimate goal here is PV backend re-load that is completely transparent to the guest. Modifying anything in xenstore compromises that so we need to be careful.

That's a fine goal, but not switching to closed state in xenbus_dev_remove seems wrong, as you have actually left the frontend without a matching backend and with the state not set to closed.

Ie: that would be fine if you explicitly state this is some kind of internal blkback reload, but not for the general case where blkback has been unbound. I think we need someway to difference a blkback reload vs a unbound.

Thanks, Roger.

#19 lore raw

RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 45 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 12:26
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Juergen Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 5 lines · Paul
Quoted 5 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 11:39
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];

Juergen

Quoted 3 lines
Quoted 3 lines

Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is

forced to

Quoted 4 lines
Quoted 4 lines

closed

Quoted 1 line · Paul Durrant

Only force state to closed in the case when the toolstack may need

to

Quoted 1 line
Quoted 1 line
Quoted 1 line

clean up. This can be detected by checking whether the state in

xenstore

Quoted 10 lines
Quoted 9 lines
Quoted 1 line

has been set to closing prior to device removal.

I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.

Can you explain what's the benefit of leaving a device without a driver in such unknown state?

Quoted 6 lines

If probe fails then I think it should leave the state alone. If the state is moved to closed then basically you just killed that connection to the guest (as the frontend will normally close down when it sees this change) so, if the probe failure was due to a bug in blkback or, e.g., a transient resource issue then it's game over as far as that guest goes.

Quoted 2 lines

But the connection can be restarted by switching the backend to the init state again.

Too late. The frontend saw closed and you already lost.

Quoted 9 lines
Quoted 2 lines

The ultimate goal here is PV backend re-load that is completely

transparent to the guest. Modifying anything in xenstore compromises that so we need to be careful.

That's a fine goal, but not switching to closed state in xenbus_dev_remove seems wrong, as you have actually left the frontend without a matching backend and with the state not set to closed.

Why is this a problem? With this series fully applied a (block) backend can come and go without needing to change the state. Relying on guests to DTRT is not a sustainable option for a cloud deployment.

Quoted 5 lines

Ie: that would be fine if you explicitly state this is some kind of internal blkback reload, but not for the general case where blkback has been unbound. I think we need someway to difference a blkback reload vs a unbound.

Why do we need that though? Why is it advantageous for a backend to go to closed. No PV backends cope with an unbind as-is, and a toolstack initiated unplug will always set state to 5 anyway. So TBH any state transition done directly in the xenbus code looks wrong to me anyway (but appears to be a necessary evil to keep the toolstack working in the event it spawns a backend where there is actually to driver present, or it doesn't come online).

Paul

#20 lore raw

Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 62 lines · Paul
Quoted 45 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 12:26
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Juergen Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 5 lines · Paul
Quoted 5 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 11:39
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];

Juergen

Quoted 3 lines
Quoted 3 lines

Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is

forced to

Quoted 4 lines
Quoted 4 lines

closed

Quoted 1 line · Paul Durrant

Only force state to closed in the case when the toolstack may need

to

Quoted 1 line
Quoted 1 line
Quoted 1 line

clean up. This can be detected by checking whether the state in

xenstore

Quoted 10 lines
Quoted 9 lines
Quoted 1 line

has been set to closing prior to device removal.

I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.

Can you explain what's the benefit of leaving a device without a driver in such unknown state?

Quoted 6 lines

If probe fails then I think it should leave the state alone. If the state is moved to closed then basically you just killed that connection to the guest (as the frontend will normally close down when it sees this change) so, if the probe failure was due to a bug in blkback or, e.g., a transient resource issue then it's game over as far as that guest goes.

Quoted 2 lines

But the connection can be restarted by switching the backend to the init state again.

Too late. The frontend saw closed and you already lost.

Quoted 9 lines
Quoted 2 lines

The ultimate goal here is PV backend re-load that is completely

transparent to the guest. Modifying anything in xenstore compromises that so we need to be careful.

That's a fine goal, but not switching to closed state in xenbus_dev_remove seems wrong, as you have actually left the frontend without a matching backend and with the state not set to closed.

Why is this a problem? With this series fully applied a (block) backend can come and go without needing to change the state. Relying on guests to DTRT is not a sustainable option for a cloud deployment.

Quoted 7 lines
Quoted 5 lines

Ie: that would be fine if you explicitly state this is some kind of internal blkback reload, but not for the general case where blkback has been unbound. I think we need someway to difference a blkback reload vs a unbound.

Why do we need that though? Why is it advantageous for a backend to go to closed. No PV backends cope with an unbind as-is, and a toolstack initiated unplug will always set state to 5 anyway. So TBH any state transition done directly in the xenbus code looks wrong to me anyway (but appears to be a necessary evil to keep the toolstack working in the event it spawns a backend where there is actually to driver present, or it doesn't come online).

IMO the normal flow for unbind would be to attempt to close open connections and then remove the driver: leaving frontends connected without any attached backends is not correct, and will just block the guest frontend until requests start timing out.

I can see the reasoning for doing that for the purpose of updating a blkback module without guests noticing, but I would prefer that leaving connections open was an option that could be given when unbinding (or maybe a driver option in sysfs?), so that the default behaviour would be to try to close everything when unbinding if possible.

Thanks, Roger.

#21 lore raw

RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 100 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 14:29
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Juergen Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 5 lines · Paul
Quoted 5 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 12:26
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];

Juergen

Quoted 3 lines
Quoted 3 lines

Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is

forced to

Quoted 10 lines
Quoted 10 lines

closed

Quoted 5 lines · Paul
Quoted 5 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 11:39
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];

Juergen

Quoted 1 line
Quoted 1 line

Gross <[email protected]>; Stefano Stabellini

<[email protected]>;

Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines

Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is

forced to

Quoted 4 lines
Quoted 4 lines

closed

Quoted 1 line · Paul Durrant

Only force state to closed in the case when the toolstack may

need

Quoted 6 lines
Quoted 6 lines

to

Quoted 1 line
Quoted 1 line
Quoted 1 line

clean up. This can be detected by checking whether the state in

xenstore

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

has been set to closing prior to device removal.

I'm not sure I see the point of this, I would expect that a

failure to

Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 6 lines

probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.

Can you explain what's the benefit of leaving a device without a driver in such unknown state?

If probe fails then I think it should leave the state alone. If the state is moved to closed then basically you just killed that connection to the guest (as the frontend will normally close down when it sees this change) so, if the probe failure was due to a bug in blkback or, e.g., a transient resource issue then it's game over as far as that guest goes.

Quoted 8 lines
Quoted 2 lines

But the connection can be restarted by switching the backend to the init state again.

Too late. The frontend saw closed and you already lost.

Quoted 3 lines
Quoted 2 lines

The ultimate goal here is PV backend re-load that is completely

transparent to the guest. Modifying anything in xenstore compromises

that

Quoted 8 lines
Quoted 6 lines

so we need to be careful.

That's a fine goal, but not switching to closed state in xenbus_dev_remove seems wrong, as you have actually left the frontend without a matching backend and with the state not set to closed.

Why is this a problem? With this series fully applied a (block) backend

can come and go without needing to change the state. Relying on guests to DTRT is not a sustainable option for a cloud deployment.

Quoted 8 lines
Quoted 6 lines

Ie: that would be fine if you explicitly state this is some kind of internal blkback reload, but not for the general case where blkback has been unbound. I think we need someway to difference a blkback reload vs a unbound.

Why do we need that though? Why is it advantageous for a backend to go

to closed. No PV backends cope with an unbind as-is, and a toolstack initiated unplug will always set state to 5 anyway. So TBH any state transition done directly in the xenbus code looks wrong to me anyway (but appears to be a necessary evil to keep the toolstack working in the event it spawns a backend where there is actually to driver present, or it doesn't come online).

IMO the normal flow for unbind would be to attempt to close open connections and then remove the driver: leaving frontends connected without any attached backends is not correct, and will just block the guest frontend until requests start timing out.

Quoted 6 lines

I can see the reasoning for doing that for the purpose of updating a blkback module without guests noticing, but I would prefer that leaving connections open was an option that could be given when unbinding (or maybe a driver option in sysfs?), so that the default behaviour would be to try to close everything when unbinding if possible.

Well unbind is pretty useless now IMO since bind doesn't work, and a transition straight to closed is just plain wrong anyway. But, we could have a flag that the backend driver sets to say that it supports transparent re-bind that gates this code. Would that make you feel more comfortable?

If you want unbind to actually do a proper unplug then that's extra work and not really something I want to tackle (and re-bind would still need to be toolstack initiated as something would have to re-create the xenstore area).

Paul

Quoted 2 lines

Thanks, Roger.

#22 lore raw

Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 100 lines · Paul
Quoted 100 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 14:29
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Juergen Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 5 lines · Paul
Quoted 5 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 12:26
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];

Juergen

Quoted 3 lines
Quoted 3 lines

Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is

forced to

Quoted 10 lines
Quoted 10 lines

closed

Quoted 5 lines · Paul
Quoted 5 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 11:39
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];

Juergen

Quoted 1 line
Quoted 1 line

Gross <[email protected]>; Stefano Stabellini

<[email protected]>;

Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines

Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is

forced to

Quoted 4 lines
Quoted 4 lines

closed

Quoted 1 line · Paul Durrant

Only force state to closed in the case when the toolstack may

need

Quoted 6 lines
Quoted 6 lines

to

Quoted 1 line
Quoted 1 line
Quoted 1 line

clean up. This can be detected by checking whether the state in

xenstore

Quoted 3 lines
Quoted 3 lines
Quoted 1 line

has been set to closing prior to device removal.

I'm not sure I see the point of this, I would expect that a

failure to

Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 6 lines

probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.

Can you explain what's the benefit of leaving a device without a driver in such unknown state?

If probe fails then I think it should leave the state alone. If the state is moved to closed then basically you just killed that connection to the guest (as the frontend will normally close down when it sees this change) so, if the probe failure was due to a bug in blkback or, e.g., a transient resource issue then it's game over as far as that guest goes.

Quoted 8 lines
Quoted 2 lines

But the connection can be restarted by switching the backend to the init state again.

Too late. The frontend saw closed and you already lost.

Quoted 3 lines
Quoted 2 lines

The ultimate goal here is PV backend re-load that is completely

transparent to the guest. Modifying anything in xenstore compromises

that

Quoted 8 lines
Quoted 6 lines

so we need to be careful.

That's a fine goal, but not switching to closed state in xenbus_dev_remove seems wrong, as you have actually left the frontend without a matching backend and with the state not set to closed.

Why is this a problem? With this series fully applied a (block) backend

can come and go without needing to change the state. Relying on guests to DTRT is not a sustainable option for a cloud deployment.

Quoted 8 lines
Quoted 6 lines

Ie: that would be fine if you explicitly state this is some kind of internal blkback reload, but not for the general case where blkback has been unbound. I think we need someway to difference a blkback reload vs a unbound.

Why do we need that though? Why is it advantageous for a backend to go

to closed. No PV backends cope with an unbind as-is, and a toolstack initiated unplug will always set state to 5 anyway. So TBH any state transition done directly in the xenbus code looks wrong to me anyway (but appears to be a necessary evil to keep the toolstack working in the event it spawns a backend where there is actually to driver present, or it doesn't come online).

IMO the normal flow for unbind would be to attempt to close open connections and then remove the driver: leaving frontends connected without any attached backends is not correct, and will just block the guest frontend until requests start timing out.

Quoted 8 lines
Quoted 6 lines

I can see the reasoning for doing that for the purpose of updating a blkback module without guests noticing, but I would prefer that leaving connections open was an option that could be given when unbinding (or maybe a driver option in sysfs?), so that the default behaviour would be to try to close everything when unbinding if possible.

Well unbind is pretty useless now IMO since bind doesn't work, and a transition straight to closed is just plain wrong anyway.

Why do you claim that a straight transition into the closed state is wrong?

I don't see any such mention in blkif.h, which also doesn't contain any guidelines regarding closing state transitions, so unless otherwise stated somewhere else transitions into closed can happen from any state IMO.

Quoted 1 line

But, we could have a flag that the backend driver sets to say that it supports transparent re-bind that gates this code. Would that make you feel more comfortable?

Having an option to leave state untouched when unbinding would be fine for me, otherwise state should be set to closed when unbinding. I don't think there's anything else that needs to be done in this regard, the cleanup should be exactly the same the only difference being the setting of all the active backends to closed state.

Quoted 1 line

If you want unbind to actually do a proper unplug then that's extra work and not really something I want to tackle (and re-bind would still need to be toolstack initiated as something would have to re-create the xenstore area).

Why do you say the xenstore area would need to be recreated?

Setting state to closed shouldn't cause any cleanup of the xenstore area, as that should already happen for example when using pvgrub since in that case grub itself disconnects and already causes a transition to closed and a re-attachment afterwards by the guest kernel.

Thanks, Roger.

#23 lore raw

RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 1 line

-----Original Message-----

[snip]

Quoted 6 lines
Quoted 2 lines

Well unbind is pretty useless now IMO since bind doesn't work, and a

transition straight to closed is just plain wrong anyway.

Why do you claim that a straight transition into the closed state is wrong?

It's badly documented, I agree, but have a look at https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/net/xen-netback/xenbus.c#n480. Connected -> Closed is not a valid transition, and I don't think it was ever intended to be.

Quoted 6 lines

I don't see any such mention in blkif.h, which also doesn't contain any guidelines regarding closing state transitions, so unless otherwise stated somewhere else transitions into closed can happen from any state IMO.

They can, but it is even more poorly documented what should be done in this case.

Quoted 10 lines
Quoted 1 line

But, we could have a flag that the backend driver sets to say that it

supports transparent re-bind that gates this code. Would that make you feel more comfortable?

Having an option to leave state untouched when unbinding would be fine for me, otherwise state should be set to closed when unbinding. I don't think there's anything else that needs to be done in this regard, the cleanup should be exactly the same the only difference being the setting of all the active backends to closed state.

Ok, I'll add such a flag and define it for blkback only, in patch #4 i.e. when it actually gains the ability to rebind.

Quoted 5 lines
Quoted 1 line

If you want unbind to actually do a proper unplug then that's extra work

and not really something I want to tackle (and re-bind would still need to be toolstack initiated as something would have to re-create the xenstore area).

Quoted 8 lines

Why do you say the xenstore area would need to be recreated?

Setting state to closed shouldn't cause any cleanup of the xenstore area, as that should already happen for example when using pvgrub since in that case grub itself disconnects and already causes a transition to closed and a re-attachment afterwards by the guest kernel.

For some reason, when I originally tested, the xenstore area disappeared. I checked again and it did not this time. I just ended up with a frontend stuck in state 5 (because it is the system disk and won't go offline) trying to talk to a non-existent backend. Upon re-bind the backend goes into state 5 (because it sees the 5 in the frontend) and leaves the guest wedged.

Paul

#24 lore raw

Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 7 lines · Paul
Quoted 7 lines
Quoted 1 line

If you want unbind to actually do a proper unplug then that's extra work

and not really something I want to tackle (and re-bind would still need to be toolstack initiated as something would have to re-create the xenstore area).

Why do you say the xenstore area would need to be recreated?

Quoted 8 lines
Quoted 6 lines

Setting state to closed shouldn't cause any cleanup of the xenstore area, as that should already happen for example when using pvgrub since in that case grub itself disconnects and already causes a transition to closed and a re-attachment afterwards by the guest kernel.

For some reason, when I originally tested, the xenstore area disappeared. I checked again and it did not this time. I just ended up with a frontend stuck in state 5 (because it is the system disk and won't go offline) trying to talk to a non-existent backend. Upon re-bind the backend goes into state 5 (because it sees the 5 in the frontend) and leaves the guest wedged.

Likely blkfront should go back to init state, but anyway, that's not something that needs fixing as part of this series.

Thanks, Roger.

#25 lore raw

RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 29 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 17:18
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Juergen Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed

Quoted 1 line · Paul
Quoted 1 line
Quoted 1 line

If you want unbind to actually do a proper unplug then that's extra

work

Quoted 1 line
Quoted 1 line

and not really something I want to tackle (and re-bind would still

need to

Quoted 1 line
Quoted 1 line

be toolstack initiated as something would have to re-create the

xenstore

Quoted 4 lines
Quoted 4 lines

area).

Why do you say the xenstore area would need to be recreated?

Quoted 8 lines
Quoted 6 lines

Setting state to closed shouldn't cause any cleanup of the xenstore area, as that should already happen for example when using pvgrub since in that case grub itself disconnects and already causes a transition to closed and a re-attachment afterwards by the guest kernel.

For some reason, when I originally tested, the xenstore area

Quoted 9 lines

disappeared. I checked again and it did not this time. I just ended up with a frontend stuck in state 5 (because it is the system disk and won't go offline) trying to talk to a non-existent backend. Upon re-bind the backend goes into state 5 (because it sees the 5 in the frontend) and leaves the guest wedged.

Likely blkfront should go back to init state, but anyway, that's not something that needs fixing as part of this series.

Ok, cool.

I am wondering though whether we ought to suppress bind/unbind for drivers that don't whitelist themselves (through the new xenbus_driver flag that I'll add). It's somewhat misleading that the nodes are there but don't necessarily work.

Paul

#26 lore raw

PATCH 3/4 xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH

In reply to Paul Durrant

Currently these macros will skip over any requests/responses that are added to the shared ring whilst it is detached. This, in general, is not a desirable semantic since most frontend implementations will eventually block waiting for a response which would either never appear or never be processed.

NOTE: These macros are currently unused. BACK_RING_ATTACH(), however, will
be used in a subsequent patch.

---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Stefano Stabellini <[email protected]>

---
 include/xen/interface/io/ring.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
include/xen/interface/io/ring.h +2 -2
index 3f40501fc60b..405adfed87e6 100644
@@ -143,14 +143,14 @@ struct __name##_back_ring {						\
 #define FRONT_RING_ATTACH(_r, _s, __size) do {				\
     (_r)->sring = (_s);							\
     (_r)->req_prod_pvt = (_s)->req_prod;				\
-    (_r)->rsp_cons = (_s)->rsp_prod;					\
+    (_r)->rsp_cons = (_s)->req_prod;					\
     (_r)->nr_ents = __RING_SIZE(_s, __size);				\
 } while (0)
 
 #define BACK_RING_ATTACH(_r, _s, __size) do {				\
     (_r)->sring = (_s);							\
     (_r)->rsp_prod_pvt = (_s)->rsp_prod;				\
-    (_r)->req_cons = (_s)->req_prod;					\
+    (_r)->req_cons = (_s)->rsp_prod;					\
     (_r)->nr_ents = __RING_SIZE(_s, __size);				\
 } while (0)
 
Signature · 1 line
2.20.1
#27 lore raw

Re: [Xen-devel] [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH

Quoted 10 lines · Paul Durrant

Currently these macros will skip over any requests/responses that are added to the shared ring whilst it is detached. This, in general, is not a desirable semantic since most frontend implementations will eventually block waiting for a response which would either never appear or never be processed.

NOTE: These macros are currently unused. BACK_RING_ATTACH(), however, will
be used in a subsequent patch.

Those headers come from Xen, and should be modified in Xen first and then imported into Linux IMO.

Thanks, Roger.

maintainer
#28 lore raw

Re: [Xen-devel] [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH

Quoted 7 lines · Roger Pau Monné
Quoted 6 lines · Paul Durrant

Currently these macros will skip over any requests/responses that are added to the shared ring whilst it is detached. This, in general, is not a desirable semantic since most frontend implementations will eventually block waiting for a response which would either never appear or never be processed.

Quoted 7 lines
Quoted 4 lines

NOTE: These macros are currently unused. BACK_RING_ATTACH(), however, will
be used in a subsequent patch.

Those headers come from Xen, and should be modified in Xen first and then imported into Linux IMO.

In theory, yes. But the Xen variant doesn't contain the ATTACH macros.

Juergen

#29 lore raw

RE: [Xen-devel] [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH

Quoted 25 lines

-----Original Message-----
From: Xen-devel <[email protected]> On Behalf Of Jürgen Groß
Sent: 09 December 2019 11:52
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; Boris Ostrovsky <[email protected]>; Stefano Stabellini <[email protected]>; [email protected]
Subject: Re: [Xen-devel] [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH

Quoted 3 lines · Roger Pau Monné
Quoted 2 lines · Paul Durrant

Currently these macros will skip over any requests/responses that are added to the shared ring whilst it is detached. This, in general, is

not

Quoted 1 line
Quoted 1 line

a desirable semantic since most frontend implementations will

eventually

Quoted 1 line
Quoted 1 line

block waiting for a response which would either never appear or never

be

Quoted 3 lines
Quoted 3 lines

processed.

NOTE: These macros are currently unused. BACK_RING_ATTACH(), however,

will

Quoted 9 lines
Quoted 6 lines
Quoted 3 lines

be used in a subsequent patch.

Those headers come from Xen, and should be modified in Xen first and then imported into Linux IMO.

In theory, yes. But the Xen variant doesn't contain the ATTACH macros.

OOI do we have a policy about this? Re-importing headers into Linux wholesale is always slightly painful because of interdependencies and style checking issues.

Paul

Quoted 7 lines

Juergen

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

maintainer
#30 lore raw

Re: [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH

In reply to Paul Durrant
Quoted 28 lines · Paul Durrant

Currently these macros will skip over any requests/responses that are added to the shared ring whilst it is detached. This, in general, is not a desirable semantic since most frontend implementations will eventually block waiting for a response which would either never appear or never be processed.

NOTE: These macros are currently unused. BACK_RING_ATTACH(), however, will
be used in a subsequent patch.

---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Stefano Stabellini <[email protected]>

---
  include/xen/interface/io/ring.h | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
include/xen/interface/io/ring.h +1 -1
index 3f40501fc60b..405adfed87e6 100644
@@ -143,14 +143,14 @@ struct __name##_back_ring {						\
  #define FRONT_RING_ATTACH(_r, _s, __size) do {				\
      (_r)->sring = (_s);							\
      (_r)->req_prod_pvt = (_s)->req_prod;				\
-    (_r)->rsp_cons = (_s)->rsp_prod;					\
+    (_r)->rsp_cons = (_s)->req_prod;					\

include/xen/interface/io/ring.h struct __name##_back_ring { \ · line 155

Quoted 10 lines
include/xen/interface/io/ring.h +1 -1
      (_r)->nr_ents = __RING_SIZE(_s, __size);				\
  } while (0)
 
  #define BACK_RING_ATTACH(_r, _s, __size) do {				\
      (_r)->sring = (_s);							\
      (_r)->rsp_prod_pvt = (_s)->rsp_prod;				\
-    (_r)->req_cons = (_s)->req_prod;					\
+    (_r)->req_cons = (_s)->rsp_prod;					\
      (_r)->nr_ents = __RING_SIZE(_s, __size);				\
  } while (0)

Lets look at all possible scenarios where BACK_RING_ATTACH() might happen:

Initially (after [FRONT|BACK]_RING_INIT(), leaving _pvt away):
req_prod=0, rsp_cons=0, rsp_prod=0, req_cons=0
Using BACK_RING_ATTACH() is fine (no change)

Request queued:
req_prod=1, rsp_cons=0, rsp_prod=0, req_cons=0
Using BACK_RING_ATTACH() is fine (no change)

and taken by backend:
req_prod=1, rsp_cons=0, rsp_prod=0, req_cons=1
Using BACK_RING_ATTACH() is resetting req_cons to 0, will result in redoing request (for blk this is fine, other devices like SCSI tapes will have issues with that). One possible solution would be to ensure all taken requests are either stopped or the response is queued already.

Response queued:
req_prod=1, rsp_cons=0, rsp_prod=1, req_cons=1
Using BACK_RING_ATTACH() is fine (no change)

Response taken:
req_prod=1, rsp_cons=1, rsp_prod=1, req_cons=1
Using BACK_RING_ATTACH() is fine (no change)

In general I believe the [FRONT|BACK]_RING_ATTACH() macros are not fine to be used in the current state, as the *_pvt fields normally not accessible by the other end are initialized using the (possibly untrusted) values from the shared ring. There needs at least to be a test for the values to be sane, and your change should not result in the same value to be read twice, as it could have changed in between.

As this is an error which can happen in other OS's, too, I'd recommend to add the adapted macros (plus a comment regarding the possible problem noted above for special devices like tapes) to the Xen variant of ring.h.

Juergen

#31 lore raw

RE: [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH

Quoted 64 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 13:55
To: Durrant, Paul <[email protected]>; [email protected]; [email protected]
Cc: Boris Ostrovsky <[email protected]>; Stefano Stabellini <[email protected]>
Subject: Re: [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH

Quoted 7 lines · Paul Durrant

Currently these macros will skip over any requests/responses that are added to the shared ring whilst it is detached. This, in general, is not a desirable semantic since most frontend implementations will eventually block waiting for a response which would either never appear or never be processed.

NOTE: These macros are currently unused. BACK_RING_ATTACH(), however,

will

Quoted 7 lines

be used in a subsequent patch.

---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Stefano Stabellini <[email protected]>

Review

Quoted 5 lines
---
  include/xen/interface/io/ring.h | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
(quoted diff) +0 -0

b/include/xen/interface/io/ring.h

include/xen/interface/io/ring.h struct __name##_back_ring {

Quoted 4 lines

index 3f40501fc60b..405adfed87e6 100644

include/xen/interface/io/ring.h +0 -0
@@ -143,14 +143,14 @@ struct __name##_back_ring {
		\
Quoted 5 lines
include/xen/interface/io/ring.h +1 -1
  #define FRONT_RING_ATTACH(_r, _s, __size) do {				\
      (_r)->sring = (_s);							\
      (_r)->req_prod_pvt = (_s)->req_prod;				\
-    (_r)->rsp_cons = (_s)->rsp_prod;					\
+    (_r)->rsp_cons = (_s)->req_prod;					\

include/xen/interface/io/ring.h struct __name##_back_ring { \ · line 155

Quoted 10 lines
include/xen/interface/io/ring.h +1 -1
      (_r)->nr_ents = __RING_SIZE(_s, __size);				\
  } while (0)
 
  #define BACK_RING_ATTACH(_r, _s, __size) do {				\
      (_r)->sring = (_s);							\
      (_r)->rsp_prod_pvt = (_s)->rsp_prod;				\
-    (_r)->req_cons = (_s)->req_prod;					\
+    (_r)->req_cons = (_s)->rsp_prod;					\
      (_r)->nr_ents = __RING_SIZE(_s, __size);				\
  } while (0)

Lets look at all possible scenarios where BACK_RING_ATTACH() might happen:

Initially (after [FRONT|BACK]_RING_INIT(), leaving _pvt away):
req_prod=0, rsp_cons=0, rsp_prod=0, req_cons=0
Using BACK_RING_ATTACH() is fine (no change)

Request queued:
req_prod=1, rsp_cons=0, rsp_prod=0, req_cons=0
Using BACK_RING_ATTACH() is fine (no change)

Quoted 7 lines

and taken by backend:
req_prod=1, rsp_cons=0, rsp_prod=0, req_cons=1
Using BACK_RING_ATTACH() is resetting req_cons to 0, will result in redoing request (for blk this is fine, other devices like SCSI tapes will have issues with that). One possible solution would be to ensure all taken requests are either stopped or the response is queued already.

Yes, it is the assumption that a backend will drain and complete any requests it is handling, but it will not deal with new ones being posted by the frontend. This does appear to be the case for blkback.

Quoted 9 lines

Response queued:
req_prod=1, rsp_cons=0, rsp_prod=1, req_cons=1
Using BACK_RING_ATTACH() is fine (no change)

Response taken:
req_prod=1, rsp_cons=1, rsp_prod=1, req_cons=1
Using BACK_RING_ATTACH() is fine (no change)

Quoted 6 lines

In general I believe the [FRONT|BACK]_RING_ATTACH() macros are not fine to be used in the current state, as the *_pvt fields normally not accessible by the other end are initialized using the (possibly untrusted) values from the shared ring. There needs at least to be a test for the values to be sane, and your change should not result in the same value to be read twice, as it could have changed in between.

What test would you apply to sanitize the value of the pvt pointer? Another option would be to have a backend write its pvt value into the xenstore backend area when the ring is unmapped, so that a new instance definitely resumes where the old one left off. The value of rsp_prod could, of course, be overwritten by the guest at any time and so there's little point in attempting sanitize it.

Quoted 6 lines

As this is an error which can happen in other OS's, too, I'd recommend to add the adapted macros (plus a comment regarding the possible problem noted above for special devices like tapes) to the Xen variant of ring.h.

I can certainly send a patch to Xen once we agree on the final definition.

Paul

Quoted 2 lines

Juergen

maintainer
#32 lore raw

Re: [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH

Quoted 83 lines · Paul
Quoted 64 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 13:55
To: Durrant, Paul <[email protected]>; [email protected]; [email protected]
Cc: Boris Ostrovsky <[email protected]>; Stefano Stabellini <[email protected]>
Subject: Re: [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH

Quoted 7 lines · Paul Durrant

Currently these macros will skip over any requests/responses that are added to the shared ring whilst it is detached. This, in general, is not a desirable semantic since most frontend implementations will eventually block waiting for a response which would either never appear or never be processed.

NOTE: These macros are currently unused. BACK_RING_ATTACH(), however,

will

Quoted 7 lines

be used in a subsequent patch.

---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Stefano Stabellini <[email protected]>

Review

Quoted 5 lines
---
   include/xen/interface/io/ring.h | 4 ++--
   1 file changed, 2 insertions(+), 2 deletions(-)
(quoted diff) +0 -0

b/include/xen/interface/io/ring.h

include/xen/interface/io/ring.h struct __name##_back_ring {

Quoted 4 lines

index 3f40501fc60b..405adfed87e6 100644

include/xen/interface/io/ring.h +0 -0
@@ -143,14 +143,14 @@ struct __name##_back_ring {
		\
Quoted 5 lines
include/xen/interface/io/ring.h +1 -1
   #define FRONT_RING_ATTACH(_r, _s, __size) do {				\
       (_r)->sring = (_s);							\
       (_r)->req_prod_pvt = (_s)->req_prod;				\
-    (_r)->rsp_cons = (_s)->rsp_prod;					\
+    (_r)->rsp_cons = (_s)->req_prod;					\

include/xen/interface/io/ring.h struct __name##_back_ring { \ · line 155

Quoted 10 lines
include/xen/interface/io/ring.h +1 -1
       (_r)->nr_ents = __RING_SIZE(_s, __size);				\
   } while (0)
 
   #define BACK_RING_ATTACH(_r, _s, __size) do {				\
       (_r)->sring = (_s);							\
       (_r)->rsp_prod_pvt = (_s)->rsp_prod;				\
-    (_r)->req_cons = (_s)->req_prod;					\
+    (_r)->req_cons = (_s)->rsp_prod;					\
       (_r)->nr_ents = __RING_SIZE(_s, __size);				\
   } while (0)

Lets look at all possible scenarios where BACK_RING_ATTACH() might happen:

Initially (after [FRONT|BACK]_RING_INIT(), leaving _pvt away):
req_prod=0, rsp_cons=0, rsp_prod=0, req_cons=0
Using BACK_RING_ATTACH() is fine (no change)

Request queued:
req_prod=1, rsp_cons=0, rsp_prod=0, req_cons=0
Using BACK_RING_ATTACH() is fine (no change)

Quoted 7 lines

and taken by backend:
req_prod=1, rsp_cons=0, rsp_prod=0, req_cons=1
Using BACK_RING_ATTACH() is resetting req_cons to 0, will result in redoing request (for blk this is fine, other devices like SCSI tapes will have issues with that). One possible solution would be to ensure all taken requests are either stopped or the response is queued already.

Yes, it is the assumption that a backend will drain and complete any requests it is handling, but it will not deal with new ones being posted by the frontend. This does appear to be the case for blkback.

Quoted 9 lines

Response queued:
req_prod=1, rsp_cons=0, rsp_prod=1, req_cons=1
Using BACK_RING_ATTACH() is fine (no change)

Response taken:
req_prod=1, rsp_cons=1, rsp_prod=1, req_cons=1
Using BACK_RING_ATTACH() is fine (no change)

Quoted 8 lines
Quoted 6 lines

In general I believe the [FRONT|BACK]_RING_ATTACH() macros are not fine to be used in the current state, as the *_pvt fields normally not accessible by the other end are initialized using the (possibly untrusted) values from the shared ring. There needs at least to be a test for the values to be sane, and your change should not result in the same value to be read twice, as it could have changed in between.

What test would you apply to sanitize the value of the pvt pointer?

For the BACK_RING_ATTACH() case rsp_prod_pvt should not be between req_prod and req_cons, and req_cons - rsp_prod_pvt should be <= ring size IMO.

Quoted 1 line

Another option would be to have a backend write its pvt value into the xenstore backend area when the ring is unmapped, so that a new instance definitely resumes where the old one left off. The value of rsp_prod could, of course, be overwritten by the guest at any time and so there's little point in attempting sanitize it.

I don't think this would be necessary. With above validation in place all the guest could do would be to shoot itself in the foot.

Juergen

#33 lore raw

PATCH 4/4 xen-blkback: support dynamic unbind/bind

In reply to Paul Durrant

By simply re-attaching to shared rings during connect_ring() rather than assuming they are freshly allocated (i.e assuming the counters are zero) it is possible for vbd instances to be unbound and re-bound from and to (respectively) a running guest.

This has been tested by running:

while true; do dd if=/dev/urandom of=test.img bs=1M count=1024; done

in a PV guest whilst running:

while true;
do echo vbd-$DOMID-$VBD >unbind;
echo unbound;
sleep 5;
echo vbd-$DOMID-$VBD >bind;
echo bound;
sleep 3;
done

in dom0 from /sys/bus/xen-backend/drivers/vbd to continuously unbind and re-bind its system disk image.

This is a highly useful feature for a backend module as it allows it to be unloaded and re-loaded (i.e. updated) without requiring domUs to be halted. This was also tested by running:

while true;
do echo vbd-$DOMID-$VBD >unbind;
echo unbound;
sleep 5;
rmmod xen-blkback;
echo unloaded;
sleep 1;
modprobe xen-blkback;
echo bound;
cd $(pwd);
sleep 3;
done

in dom0 whilst running the same loop as above in the (single) PV guest.

Some (less stressful) testing has also been done using a Windows HVM guest with the latest 9.0 PV drivers installed.

---
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
Cc: Jens Axboe <[email protected]>
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Stefano Stabellini <[email protected]>

---
 drivers/block/xen-blkback/xenbus.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/block/xen-blkback/xenbus.c +6 -6
index e8c5c54e1d26..0b82740c4a9d 100644
@@ -196,24 +196,24 @@ static int xen_blkif_map(struct xen_blkif_ring *ring, grant_ref_t *gref,
 	{
 		struct blkif_sring *sring;
 		sring = (struct blkif_sring *)ring->blk_ring;
-		BACK_RING_INIT(&ring->blk_rings.native, sring,
-			       XEN_PAGE_SIZE * nr_grefs);
+		BACK_RING_ATTACH(&ring->blk_rings.native, sring,
+				 XEN_PAGE_SIZE * nr_grefs);
 		break;
 	}
 	case BLKIF_PROTOCOL_X86_32:
 	{
 		struct blkif_x86_32_sring *sring_x86_32;
 		sring_x86_32 = (struct blkif_x86_32_sring *)ring->blk_ring;
-		BACK_RING_INIT(&ring->blk_rings.x86_32, sring_x86_32,
-			       XEN_PAGE_SIZE * nr_grefs);
+		BACK_RING_ATTACH(&ring->blk_rings.x86_32, sring_x86_32,
+				 XEN_PAGE_SIZE * nr_grefs);
 		break;
 	}
 	case BLKIF_PROTOCOL_X86_64:
 	{
 		struct blkif_x86_64_sring *sring_x86_64;
 		sring_x86_64 = (struct blkif_x86_64_sring *)ring->blk_ring;
-		BACK_RING_INIT(&ring->blk_rings.x86_64, sring_x86_64,
-			       XEN_PAGE_SIZE * nr_grefs);
+		BACK_RING_ATTACH(&ring->blk_rings.x86_64, sring_x86_64,
+				 XEN_PAGE_SIZE * nr_grefs);
 		break;
 	}
 	default:
Signature · 1 line
2.20.1
#34 lore raw

Re: [PATCH 4/4] xen-blkback: support dynamic unbind/bind

Quoted 11 lines · Paul Durrant

By simply re-attaching to shared rings during connect_ring() rather than assuming they are freshly allocated (i.e assuming the counters are zero) it is possible for vbd instances to be unbound and re-bound from and to (respectively) a running guest.

This has been tested by running:

while true; do dd if=/dev/urandom of=test.img bs=1M count=1024; done

in a PV guest whilst running:

Quoted 8 lines

while true;
do echo vbd-$DOMID-$VBD >unbind;
echo unbound;
sleep 5;
echo vbd-$DOMID-$VBD >bind;
echo bound;
sleep 3;
done

So this does unbind blkback while leaving the PV interface as connected?

Thanks, Roger.

#35 lore raw

RE: [PATCH 4/4] xen-blkback: support dynamic unbind/bind

Quoted 31 lines

-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 12:17
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Konrad Rzeszutek Wilk <[email protected]>; Jens Axboe <[email protected]>; Boris Ostrovsky <[email protected]>; Juergen Gross <[email protected]>; Stefano Stabellini <[email protected]>
Subject: Re: [PATCH 4/4] xen-blkback: support dynamic unbind/bind

Quoted 11 lines · Paul Durrant

By simply re-attaching to shared rings during connect_ring() rather than assuming they are freshly allocated (i.e assuming the counters are zero) it is possible for vbd instances to be unbound and re-bound from and to (respectively) a running guest.

This has been tested by running:

while true; do dd if=/dev/urandom of=test.img bs=1M count=1024; done

in a PV guest whilst running:

Quoted 8 lines

while true;
do echo vbd-$DOMID-$VBD >unbind;
echo unbound;
sleep 5;
echo vbd-$DOMID-$VBD >bind;
echo bound;
sleep 3;
done

Quoted 3 lines

So this does unbind blkback while leaving the PV interface as connected?

Yes, everything is left in place in the frontend. The backend detaches from the ring, closes its end of the event channels, etc. but the guest can still send requests which will get serviced when the new backend attaches.

Paul

maintainer
#36 lore raw

Re: [PATCH 4/4] xen-blkback: support dynamic unbind/bind

In reply to Paul Durrant
Quoted 20 lines · Paul Durrant

By simply re-attaching to shared rings during connect_ring() rather than assuming they are freshly allocated (i.e assuming the counters are zero) it is possible for vbd instances to be unbound and re-bound from and to (respectively) a running guest.

This has been tested by running:

while true; do dd if=/dev/urandom of=test.img bs=1M count=1024; done

in a PV guest whilst running:

while true;
   do echo vbd-$DOMID-$VBD >unbind;
   echo unbound;
   sleep 5;
   echo vbd-$DOMID-$VBD >bind;
   echo bound;
   sleep 3;
   done
Quoted 2 lines

in dom0 from /sys/bus/xen-backend/drivers/vbd to continuously unbind and re-bind its system disk image.

Could you do the same test with mixed reads/writes and verification of the read/written data, please? A write-only test is not _that_ convincing regarding correctness. It only proves the guest is not crashing.

I'm fine with the general approach, though.

Juergen

#37 lore raw

RE: [PATCH 4/4] xen-blkback: support dynamic unbind/bind

Quoted 31 lines

-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 13:58
To: Durrant, Paul <[email protected]>; [email protected]; [email protected]
Cc: Konrad Rzeszutek Wilk <[email protected]>; Roger Pau Monné <[email protected]>; Jens Axboe <[email protected]>; Boris Ostrovsky <[email protected]>; Stefano Stabellini <[email protected]>
Subject: Re: [PATCH 4/4] xen-blkback: support dynamic unbind/bind

Quoted 20 lines · Paul Durrant

By simply re-attaching to shared rings during connect_ring() rather than assuming they are freshly allocated (i.e assuming the counters are zero) it is possible for vbd instances to be unbound and re-bound from and to (respectively) a running guest.

This has been tested by running:

while true; do dd if=/dev/urandom of=test.img bs=1M count=1024; done

in a PV guest whilst running:

while true;
   do echo vbd-$DOMID-$VBD >unbind;
   echo unbound;
   sleep 5;
   echo vbd-$DOMID-$VBD >bind;
   echo bound;
   sleep 3;
   done
Quoted 7 lines
Quoted 2 lines

in dom0 from /sys/bus/xen-backend/drivers/vbd to continuously unbind and re-bind its system disk image.

Could you do the same test with mixed reads/writes and verification of the read/written data, please? A write-only test is not _that_ convincing regarding correctness. It only proves the guest is not crashing.

Sure. I'll find something that will verify content.

Quoted 3 lines

I'm fine with the general approach, though.

Cool, thanks,

Paul

Quoted 2 lines

Juergen