xen-blkback: support live update
Subsystems (3)
cover xen-blkback: support live update
Patch #1 is clean-up for an apparent mis-feature.
Paul Durrant (4):
xenbus: move xenbus_dev_shutdown() into frontend code...
xenbus: limit when state is forced to closed xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH
xen-blkback: support dynamic unbind/bind
drivers/block/xen-blkback/xenbus.c | 12 ++++---- drivers/xen/xenbus/xenbus.h | 2 -- drivers/xen/xenbus/xenbus_probe.c | 34 ++++++---------------- drivers/xen/xenbus/xenbus_probe_backend.c | 1 - drivers/xen/xenbus/xenbus_probe_frontend.c | 24 ++++++++++++++- include/xen/interface/io/ring.h | 4 +-- 6 files changed, 40 insertions(+), 37 deletions(-)
---
Cc: Boris Ostrovsky <[email protected]>
Cc: Jens Axboe <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
Cc: Stefano Stabellini <[email protected]>
Signature · 1 line
2.20.1
PATCH 1/4 xenbus: move xenbus_dev_shutdown() into frontend code...
...and make it static
xenbus_dev_shutdown() is seemingly intended to cause clean shutdown of PV frontends when a guest is rebooted. Indeed the function waits for a conpletion which is only set by a call to xenbus_frontend_closed().
This patch removes the shutdown() method from backends and moves xenbus_dev_shutdown() from xenbus_probe.c into xenbus_probe_frontend.c, renaming it appropriately and making it static.
---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Stefano Stabellini <[email protected]>
--- drivers/xen/xenbus/xenbus.h | 2 -- drivers/xen/xenbus/xenbus_probe.c | 23 --------------------- drivers/xen/xenbus/xenbus_probe_backend.c | 1 - drivers/xen/xenbus/xenbus_probe_frontend.c | 24 +++++++++++++++++++++- 4 files changed, 23 insertions(+), 27 deletions(-)
drivers/xen/xenbus/xenbus.h +0 -2
@@ -116,8 +116,6 @@ int xenbus_probe_devices(struct xen_bus_type *bus);void xenbus_dev_changed (const char *node,struct xen_bus_type *bus); -void xenbus_dev_shutdown (struct device *_dev); -int xenbus_dev_suspend (struct device *dev);int xenbus_dev_resume (struct device *dev);int xenbus_dev_cancel (struct device *dev);
drivers/xen/xenbus/xenbus_probe.c +0 -23
@@ -281,29 +281,6 @@ int xenbus_dev_remove(struct device *_dev) }EXPORT_SYMBOL_GPL (xenbus_dev_remove); -void xenbus_dev_shutdown (struct device *_dev) -{ -struct xenbus_device *dev =to_xenbus_device (_dev); -unsigned long timeout =5 *HZ ; - -DPRINTK ("%s" , dev->nodename); - -get_device (&dev->dev); -if (dev->state != XenbusStateConnected) { -pr_info ("%s: %s: %s != Connected, skipping\n" , - __func__, dev->nodename,xenbus_strstate (dev->state)); - goto out; - } -xenbus_switch_state (dev, XenbusStateClosing); - timeout =wait_for_completion_timeout (&dev->down, timeout); -if (!timeout) -pr_info ("%s: %s timeout closing device\n" , - __func__, dev->nodename); - out: -put_device (&dev->dev); -} -EXPORT_SYMBOL_GPL (xenbus_dev_shutdown); -int xenbus_register_driver_common (struct xenbus_driver *drv,struct xen_bus_type *bus,struct module *owner,const char *mod_name)
drivers/xen/xenbus/xenbus_probe_backend.c +0 -1
@@ -198,7 +198,6 @@ static struct xen_bus_type xenbus_backend = { .uevent = xenbus_uevent_backend, .probe = xenbus_dev_probe, .remove = xenbus_dev_remove, - .shutdown = xenbus_dev_shutdown, .dev_groups = xenbus_dev_groups, }, };
drivers/xen/xenbus/xenbus_probe_frontend.c +23 -1
@@ -126,6 +126,28 @@ static int xenbus_frontend_dev_probe(struct device *dev)return xenbus_dev_probe (dev); } +static void xenbus_frontend_dev_shutdown (struct device *_dev) +{ +struct xenbus_device *dev =to_xenbus_device (_dev); +unsigned long timeout =5 *HZ ; + +DPRINTK ("%s" , dev->nodename); + +get_device (&dev->dev); +if (dev->state != XenbusStateConnected) { +pr_info ("%s: %s: %s != Connected, skipping\n" , + __func__, dev->nodename,xenbus_strstate (dev->state)); + goto out; + } +xenbus_switch_state (dev, XenbusStateClosing); + timeout =wait_for_completion_timeout (&dev->down, timeout); +if (!timeout) +pr_info ("%s: %s timeout closing device\n" , + __func__, dev->nodename); + out: +put_device (&dev->dev); +} +static const struct dev_pm_ops xenbus_pm_ops = { .suspend = xenbus_dev_suspend, .resume = xenbus_frontend_dev_resume,
@@ -146,7 +168,7 @@ static struct xen_bus_type xenbus_frontend = { .uevent = xenbus_uevent_frontend, .probe = xenbus_frontend_dev_probe, .remove = xenbus_dev_remove, - .shutdown = xenbus_dev_shutdown, + .shutdown = xenbus_frontend_dev_shutdown, .dev_groups = xenbus_dev_groups, .pm = &xenbus_pm_ops,
Signature · 1 line
2.20.1
Jürgen Groß
maintainerRe: [PATCH 1/4] xenbus: move xenbus_dev_shutdown() into frontend code...
Quoted 9 lines · Paul Durrant
...and make it static
xenbus_dev_shutdown()is seemingly intended to cause clean shutdown of PV frontends when a guest is rebooted. Indeed the function waits for a conpletion which is only set by a call toxenbus_frontend_closed().This patch removes the
shutdown()method from backends and movesxenbus_dev_shutdown()fromxenbus_probe.cintoxenbus_probe_frontend.c, renaming it appropriately and making it static.
Is this a good move considering driver domains?
At least I'd expect the commit message addressing the expected behavior with rebooting a driver domain and why this patch isn't making things worse.
Juergen
RE: [PATCH 1/4] xenbus: move xenbus_dev_shutdown() into frontend code...
Quoted 15 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:34
To: Durrant, Paul <[email protected]>; [email protected]; [email protected]
Cc: Boris Ostrovsky <[email protected]>; Stefano Stabellini <[email protected]>
Subject: Re: [PATCH 1/4] xenbus: movexenbus_dev_shutdown()into frontend code...Quoted 3 lines · Paul Durrant
...and make it static
xenbus_dev_shutdown()is seemingly intended to cause clean shutdown ofPV
Quoted 8 lines
Quoted 6 lines
frontends when a guest is rebooted. Indeed the function waits for a conpletion which is only set by a call to
xenbus_frontend_closed().This patch removes the
shutdown()method from backends and movesxenbus_dev_shutdown()fromxenbus_probe.cintoxenbus_probe_frontend.c, renaming it appropriately and making it static.Is this a good move considering driver domains?
I don't think it can have ever worked properly for driver domains, and with the rest of the patches a backend should be able go away and return unannounced (as long as the domain id is kept... for which patches need to be upstreamed into Xen).
Quoted 5 lines
At least I'd expect the commit message addressing the expected behavior with rebooting a driver domain and why this patch isn't making things worse.
For a clean reboot I'd expect the toolstack to shut down the protocol before rebooting the driver domain, so the backend shutdown method is moot. And I don't believe re-startable driver domains were something that ever made it into support (because of the non-persistent domid problem). I can add something to the commit comment to that effect if you'd like.
Paul
Re: [PATCH 1/4] xenbus: move xenbus_dev_shutdown() into frontend code...
Quoted 27 lines · Paul
Quoted 15 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:34
To: Durrant, Paul <[email protected]>; [email protected]; [email protected]
Cc: Boris Ostrovsky <[email protected]>; Stefano Stabellini <[email protected]>
Subject: Re: [PATCH 1/4] xenbus: movexenbus_dev_shutdown()into frontend code...Quoted 3 lines · Paul Durrant
...and make it static
xenbus_dev_shutdown()is seemingly intended to cause clean shutdown ofPV
Quoted 8 lines
Quoted 6 lines
frontends when a guest is rebooted. Indeed the function waits for a conpletion which is only set by a call to
xenbus_frontend_closed().This patch removes the
shutdown()method from backends and movesxenbus_dev_shutdown()fromxenbus_probe.cintoxenbus_probe_frontend.c, renaming it appropriately and making it static.Is this a good move considering driver domains?
I don't think it can have ever worked properly for driver domains, and with the rest of the patches a backend should be able go away and return unannounced (as long as the domain id is kept... for which patches need to be upstreamed into Xen).
Quoted 1 line
Quoted 6 lines
Quoted 4 lines
At least I'd expect the commit message addressing the expected behavior with rebooting a driver domain and why this patch isn't making things worse.
For a clean reboot I'd expect the toolstack to shut down the protocol before rebooting the driver domain, so the backend shutdown method is moot. And I don't believe re-startable driver domains were something that ever made it into support (because of the non-persistent domid problem). I can add something to the commit comment to that effect if you'd like.
Yes, please do so.
With this you can add my:
Juergen
PATCH 2/4 xenbus: limit when state is forced to closed
In reply to Paul DurrantOnly force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in xenstore has been set to closing prior to device removal.
---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Stefano Stabellini <[email protected]>
--- drivers/xen/xenbus/xenbus_probe.c | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-)
drivers/xen/xenbus/xenbus_probe.c +9 -2
@@ -255,7 +255,6 @@ int xenbus_dev_probe(struct device *_dev)module_put (drv->driver.owner); fail:xenbus_dev_error (dev, err,"xenbus_dev_probe on %s" , dev->nodename); -xenbus_switch_state (dev, XenbusStateClosed);return err; }EXPORT_SYMBOL_GPL (xenbus_dev_probe);
@@ -264,6 +263,7 @@ int xenbus_dev_remove(struct device *_dev) {struct xenbus_device *dev =to_xenbus_device (_dev);struct xenbus_driver *drv =to_xenbus_driver (_dev->driver); +enum xenbus_state state;DPRINTK ("%s" , dev->nodename);
@@ -276,7 +276,14 @@ int xenbus_dev_remove(struct device *_dev)free_otherend_details (dev); -xenbus_switch_state (dev, XenbusStateClosed); +/* +* If the toolstack had force the device state to closing then set +* the state to closed now to allow it to be cleaned up. +*/ + state =xenbus_read_driver_state (dev->nodename); +if (state == XenbusStateClosing) +xenbus_switch_state (dev, XenbusStateClosed); +return 0 ; }EXPORT_SYMBOL_GPL (xenbus_dev_remove);
Signature · 1 line
2.20.1
Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 3 lines · Paul Durrant
Only force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in xenstore has been set to closing prior to device removal.
I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.
Can you explain what's the benefit of leaving a device without a driver in such unknown state?
Thanks, Roger.
Jürgen Groß
maintainerRe: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 11 lines · Roger Pau Monné
Quoted 3 lines · Paul Durrant
Only force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in xenstore has been set to closing prior to device removal.
I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.
Can you explain what's the benefit of leaving a device without a driver in such unknown state?
And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having been set before?
Juergen
RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 22 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closedQuoted 3 lines · Roger Pau Monné
Quoted 2 lines · Paul Durrant
Only force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in
xenstore
Quoted 6 lines
Quoted 1 line
has been set to closing prior to device removal.
I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.
Quoted 6 lines
Quoted 2 lines
Can you explain what's the benefit of leaving a device without a driver in such unknown state?
And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having been set before?
Blkfront doesn't seem to mind and I believe the Windows PV drivers cope, but I don't really understand the comment since this patch is actually removing a case where the backend transitions directly to closed.
Paul
Quoted 3 lines
Juergen
Jürgen Groß
maintainerRe: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 22 lines · Paul
Quoted 22 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closedQuoted 3 lines · Roger Pau Monné
Quoted 2 lines · Paul Durrant
Only force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in
xenstore
Quoted 6 lines
Quoted 1 line
has been set to closing prior to device removal.
I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Can you explain what's the benefit of leaving a device without a driver in such unknown state?
And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having been set before?
Blkfront doesn't seem to mind and I believe the Windows PV drivers cope, but I don't really understand the comment since this patch is actually removing a case where the backend transitions directly to closed.
I'm not speaking of blkfront/blkback only, but of net, tpm, scsi, pvcall etc. frontends/backends. After all you are modifying a function common to all PV driver pairs.
You are removing a state switc to "closed" in case the state was _not_ "closing" before. So any PV driver reacting to "closed" of the other end in case the previous state might not have been "closing" before is at risk to misbehave with your patch.
Juergen
RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 47 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 12:09
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closedQuoted 6 lines · Paul
Quoted 6 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];Stefano
Quoted 3 lines
Quoted 3 lines
Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forcedto
Quoted 13 lines
Quoted 13 lines
closed
Quoted 3 lines · Roger Pau Monné
Quoted 2 lines · Paul Durrant
Only force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in
xenstore
Quoted 6 lines
Quoted 1 line
has been set to closing prior to device removal.
I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Can you explain what's the benefit of leaving a device without a driver in such unknown state?
And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having been set before?
Blkfront doesn't seem to mind and I believe the Windows PV drivers cope,
but I don't really understand the comment since this patch is actually removing a case where the backend transitions directly to closed.
Quoted 6 lines
I'm not speaking of blkfront/blkback only, but of net, tpm, scsi, pvcall etc. frontends/backends. After all you are modifying a function common to all PV driver pairs.
You are removing a state switc to "closed" in case the state was _not_ "closing" before.
Yes, which AFAIK is against the intention of the generic PV protocol such that it ever existed anyway.
Quoted 3 lines
So any PV driver reacting to "closed" of the other end
in case the previous state might not have been "closing" before is at risk to misbehave with your patch.
Well, they will see nothing now. If the state was not closing, it gets left alone, so the frontend shouldn't do anything. The only risk that I can see is that some frontend/backend pair needed a direct 4 -> 6 transition to support 'unbind' before but AFAIK nothing has ever supported that, and blk and net crash'n'burn if you try that on upstream as it stands. A clean unplug would always set state to 5 first, since that's part of the unplug protocol.
Paul
Quoted 2 lines
Juergen
Jürgen Groß
maintainerRe: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 47 lines · Paul
Quoted 47 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 12:09
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closedQuoted 6 lines · Paul
Quoted 6 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];Stefano
Quoted 3 lines
Quoted 3 lines
Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forcedto
Quoted 13 lines
Quoted 13 lines
closed
Quoted 3 lines · Roger Pau Monné
Quoted 2 lines · Paul Durrant
Only force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in
xenstore
Quoted 6 lines
Quoted 1 line
has been set to closing prior to device removal.
I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Can you explain what's the benefit of leaving a device without a driver in such unknown state?
And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having been set before?
Blkfront doesn't seem to mind and I believe the Windows PV drivers cope,
but I don't really understand the comment since this patch is actually removing a case where the backend transitions directly to closed.
Quoted 8 lines
Quoted 6 lines
I'm not speaking of blkfront/blkback only, but of net, tpm, scsi, pvcall etc. frontends/backends. After all you are modifying a function common to all PV driver pairs.
You are removing a state switc to "closed" in case the state was _not_ "closing" before.
Yes, which AFAIK is against the intention of the generic PV protocol such that it ever existed anyway.
While this might be the case we should _not_ break any guests running now. So this kind of reasoning is dangerous.
Quoted 6 lines
Quoted 4 lines
So any PV driver reacting to "closed" of the other end
in case the previous state might not have been "closing" before is at risk to misbehave with your patch.Well, they will see nothing now. If the state was not closing, it gets left alone, so the frontend shouldn't do anything. The only risk that I can see is that some frontend/backend pair needed a direct 4 -> 6 transition to support 'unbind' before but AFAIK nothing has ever supported that, and blk and net crash'n'burn if you try that on upstream as it stands. A clean unplug would always set state to 5 first, since that's part of the unplug protocol.
That was my question: are you sure all current and previous guest frontends and backends are handling unplug this way?
Not "should handle", but "do handle".
Juergen
RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 92 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 13:39
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closedQuoted 6 lines · Paul
Quoted 6 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 12:09
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];Stefano
Quoted 3 lines
Quoted 3 lines
Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forcedto
Quoted 9 lines
Quoted 9 lines
closed
Quoted 6 lines · Paul
Quoted 6 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];Quoted 4 lines
Quoted 4 lines
Stefano
Quoted 3 lines
Quoted 3 lines
Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state isforced
Quoted 6 lines
Quoted 6 lines
to
Quoted 5 lines
Quoted 5 lines
closed
Quoted 2 lines · Roger Pau Monné
Quoted 1 line · Paul Durrant
Only force state to closed in the case when the toolstack may need
to
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
clean up. This can be detected by checking whether the state in
xenstore
Quoted 3 lines
Quoted 1 line
has been set to closing prior to device removal.
I'm not sure I see the point of this, I would expect that a failure
to
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 9 lines
Quoted 5 lines
probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.
Can you explain what's the benefit of leaving a device without a driver in such unknown state?
And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having been set before?
Blkfront doesn't seem to mind and I believe the Windows PV drivers
cope,
Quoted 4 lines
Quoted 4 lines
but I don't really understand the comment since this patch is actually removing a case where the backend transitions directly to closed.
I'm not speaking of blkfront/blkback only, but of net, tpm, scsi,
pvcall
Quoted 7 lines
Quoted 5 lines
etc. frontends/backends. After all you are modifying a function common to all PV driver pairs.
You are removing a state switc to "closed" in case the state was _not_ "closing" before.
Yes, which AFAIK is against the intention of the generic PV protocol
such that it ever existed anyway.
While this might be the case we should _not_ break any guests running now. So this kind of reasoning is dangerous.
Quoted 6 lines
Quoted 4 lines
So any PV driver reacting to "closed" of the other end
in case the previous state might not have been "closing" before is at risk to misbehave with your patch.Well, they will see nothing now. If the state was not closing, it gets
left alone, so the frontend shouldn't do anything. The only risk that I can see is that some frontend/backend pair needed a direct 4 -> 6 transition to support 'unbind' before but AFAIK nothing has ever supported that, and blk and net crash'n'burn if you try that on upstream as it stands. A clean unplug would always set state to 5 first, since that's part of the unplug protocol.
Quoted 4 lines
That was my question: are you sure all current and previous guest frontends and backends are handling unplug this way?
Not "should handle", but "do handle".
That depends on the toolstack. IIUC the only 'supported' toolstack is xl/libxl, which will set 'state' to 5 and 'online' to 0 to initiate an unplug.
Paul
Quoted 3 lines
Juergen
Jürgen Groß
maintainerRe: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 92 lines · Paul
Quoted 92 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 13:39
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closedQuoted 6 lines · Paul
Quoted 6 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 12:09
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];Stefano
Quoted 3 lines
Quoted 3 lines
Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forcedto
Quoted 9 lines
Quoted 9 lines
closed
Quoted 6 lines · Paul
Quoted 6 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];Quoted 4 lines
Quoted 4 lines
Stefano
Quoted 3 lines
Quoted 3 lines
Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state isforced
Quoted 6 lines
Quoted 6 lines
to
Quoted 5 lines
Quoted 5 lines
closed
Quoted 2 lines · Roger Pau Monné
Quoted 1 line · Paul Durrant
Only force state to closed in the case when the toolstack may need
to
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
clean up. This can be detected by checking whether the state in
xenstore
Quoted 3 lines
Quoted 1 line
has been set to closing prior to device removal.
I'm not sure I see the point of this, I would expect that a failure
to
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 9 lines
Quoted 5 lines
probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.
Can you explain what's the benefit of leaving a device without a driver in such unknown state?
And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having been set before?
Blkfront doesn't seem to mind and I believe the Windows PV drivers
cope,
Quoted 4 lines
Quoted 4 lines
but I don't really understand the comment since this patch is actually removing a case where the backend transitions directly to closed.
I'm not speaking of blkfront/blkback only, but of net, tpm, scsi,
pvcall
Quoted 7 lines
Quoted 5 lines
etc. frontends/backends. After all you are modifying a function common to all PV driver pairs.
You are removing a state switc to "closed" in case the state was _not_ "closing" before.
Yes, which AFAIK is against the intention of the generic PV protocol
such that it ever existed anyway.
While this might be the case we should _not_ break any guests running now. So this kind of reasoning is dangerous.
Quoted 6 lines
Quoted 4 lines
So any PV driver reacting to "closed" of the other end
in case the previous state might not have been "closing" before is at risk to misbehave with your patch.Well, they will see nothing now. If the state was not closing, it gets
left alone, so the frontend shouldn't do anything. The only risk that I can see is that some frontend/backend pair needed a direct 4 -> 6 transition to support 'unbind' before but AFAIK nothing has ever supported that, and blk and net crash'n'burn if you try that on upstream as it stands. A clean unplug would always set state to 5 first, since that's part of the unplug protocol.
Quoted 6 lines
Quoted 4 lines
That was my question: are you sure all current and previous guest frontends and backends are handling unplug this way?
Not "should handle", but "do handle".
That depends on the toolstack. IIUC the only 'supported' toolstack is xl/libxl, which will set 'state' to 5 and 'online' to 0 to initiate an unplug.
I guess libvirt/libxl is doing the same?
At least at SUSE we still have some customers running xend based Xen installations with recent Linux or Windows guests.
Juergen
RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 117 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 14:10
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closedQuoted 6 lines · Paul
Quoted 6 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 13:39
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];Stefano
Quoted 3 lines
Quoted 3 lines
Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forcedto
Quoted 9 lines
Quoted 9 lines
closed
Quoted 6 lines · Paul
Quoted 6 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 12:09
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];Quoted 4 lines
Quoted 4 lines
Stefano
Quoted 3 lines
Quoted 3 lines
Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state isforced
Quoted 14 lines
Quoted 14 lines
to
Quoted 13 lines
Quoted 13 lines
closed
Quoted 6 lines · Paul
Quoted 6 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];Stefano
Quoted 3 lines
Quoted 3 lines
Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state isQuoted 7 lines
Quoted 7 lines
forced
Quoted 6 lines
Quoted 6 lines
to
Quoted 5 lines
Quoted 5 lines
closed
Quoted 2 lines · Roger Pau Monné
Quoted 1 line · Paul Durrant
Only force state to closed in the case when the toolstack may
need
Quoted 6 lines
Quoted 6 lines
to
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
clean up. This can be detected by checking whether the state in
xenstore
Quoted 3 lines
Quoted 1 line
has been set to closing prior to device removal.
I'm not sure I see the point of this, I would expect that a
failure
Quoted 9 lines
Quoted 9 lines
to
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.
Can you explain what's the benefit of leaving a device without a driver in such unknown state?
And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having
been
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
set before?
Blkfront doesn't seem to mind and I believe the Windows PV drivers
cope,
Quoted 1 line
Quoted 1 line
but I don't really understand the comment since this patch is
actually
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
removing a case where the backend transitions directly to closed.
I'm not speaking of blkfront/blkback only, but of net, tpm, scsi,
pvcall
Quoted 1 line
Quoted 1 line
etc. frontends/backends. After all you are modifying a function
common
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
to all PV driver pairs.
You are removing a state switc to "closed" in case the state was
_not_
Quoted 9 lines
Quoted 9 lines
Quoted 3 lines
Quoted 1 line
"closing" before.
Yes, which AFAIK is against the intention of the generic PV protocol
such that it ever existed anyway.
While this might be the case we should _not_ break any guests running now. So this kind of reasoning is dangerous.
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
So any PV driver reacting to "closed" of the other end
in case the previous state might not have been "closing" before is at risk to misbehave with your patch.Well, they will see nothing now. If the state was not closing, it gets
left alone, so the frontend shouldn't do anything. The only risk that I can see is that some frontend/backend pair needed a direct 4 -> 6 transition to support 'unbind' before but AFAIK nothing has ever
supported
Quoted 7 lines
Quoted 7 lines
that, and blk and net crash'n'burn if you try that on upstream as it stands. A clean unplug would always set state to 5 first, since that's part of the unplug protocol.
That was my question: are you sure all current and previous guest frontends and backends are handling unplug this way?
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
Not "should handle", but "do handle".
That depends on the toolstack. IIUC the only 'supported' toolstack is
xl/libxl, which will set 'state' to 5 and 'online' to 0 to initiate an unplug.
I guess libvirt/libxl is doing the same?
The unplug mechansism is all in libxl AFAICT, so it should be identical.
Quoted 3 lines
At least at SUSE we still have some customers running xend based Xen installations with recent Linux or Windows guests.
Is that something the upstream code can/should support though? I'd be surprised if xend is actually doing anything different to libxl since I've been coding the Windows PV drivers to trigger off the combined closing/online transition for as long as I can remember.
Paul
Jürgen Groß
maintainerRe: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 126 lines · Paul
Quoted 117 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 14:10
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closedQuoted 6 lines · Paul
Quoted 6 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 13:39
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];Stefano
Quoted 3 lines
Quoted 3 lines
Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forcedto
Quoted 9 lines
Quoted 9 lines
closed
Quoted 6 lines · Paul
Quoted 6 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 12:09
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];Quoted 4 lines
Quoted 4 lines
Stefano
Quoted 3 lines
Quoted 3 lines
Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state isforced
Quoted 14 lines
Quoted 14 lines
to
Quoted 13 lines
Quoted 13 lines
closed
Quoted 6 lines · Paul
Quoted 6 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];Stefano
Quoted 3 lines
Quoted 3 lines
Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state isQuoted 7 lines
Quoted 7 lines
forced
Quoted 6 lines
Quoted 6 lines
to
Quoted 5 lines
Quoted 5 lines
closed
Quoted 2 lines · Roger Pau Monné
Quoted 1 line · Paul Durrant
Only force state to closed in the case when the toolstack may
need
Quoted 6 lines
Quoted 6 lines
to
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
clean up. This can be detected by checking whether the state in
xenstore
Quoted 3 lines
Quoted 1 line
has been set to closing prior to device removal.
I'm not sure I see the point of this, I would expect that a
failure
Quoted 9 lines
Quoted 9 lines
to
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.
Can you explain what's the benefit of leaving a device without a driver in such unknown state?
And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having
been
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
set before?
Blkfront doesn't seem to mind and I believe the Windows PV drivers
cope,
Quoted 1 line
Quoted 1 line
but I don't really understand the comment since this patch is
actually
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
removing a case where the backend transitions directly to closed.
I'm not speaking of blkfront/blkback only, but of net, tpm, scsi,
pvcall
Quoted 1 line
Quoted 1 line
etc. frontends/backends. After all you are modifying a function
common
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
to all PV driver pairs.
You are removing a state switc to "closed" in case the state was
_not_
Quoted 9 lines
Quoted 9 lines
Quoted 3 lines
Quoted 1 line
"closing" before.
Yes, which AFAIK is against the intention of the generic PV protocol
such that it ever existed anyway.
While this might be the case we should _not_ break any guests running now. So this kind of reasoning is dangerous.
Quoted 1 line
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
So any PV driver reacting to "closed" of the other end
in case the previous state might not have been "closing" before is at risk to misbehave with your patch.Well, they will see nothing now. If the state was not closing, it gets
left alone, so the frontend shouldn't do anything. The only risk that I can see is that some frontend/backend pair needed a direct 4 -> 6 transition to support 'unbind' before but AFAIK nothing has ever
supported
Quoted 7 lines
Quoted 7 lines
that, and blk and net crash'n'burn if you try that on upstream as it stands. A clean unplug would always set state to 5 first, since that's part of the unplug protocol.
That was my question: are you sure all current and previous guest frontends and backends are handling unplug this way?
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
Not "should handle", but "do handle".
That depends on the toolstack. IIUC the only 'supported' toolstack is
xl/libxl, which will set 'state' to 5 and 'online' to 0 to initiate an unplug.
I guess libvirt/libxl is doing the same?
Quoted 7 lines
The unplug mechansism is all in libxl AFAICT, so it should be identical.
Quoted 3 lines
At least at SUSE we still have some customers running xend based Xen installations with recent Linux or Windows guests.
Is that something the upstream code can/should support though? I'd be surprised if xend is actually doing anything different to libxl since I've been coding the Windows PV drivers to trigger off the combined closing/online transition for as long as I can remember.
I'd rather not have to carry a private patch for new Linux kernel to be able to run on those hosts.
AFAIK you at Amazon have some quite old Xen installations, too. How are you handling that (assuming the customer is updating the kernel to a recent version in his guest)?
Juergen
RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 157 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 14:41
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected]; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closedQuoted 6 lines · Paul
Quoted 6 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 14:10
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];Stefano
Quoted 3 lines
Quoted 3 lines
Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forcedto
Quoted 9 lines
Quoted 9 lines
closed
Quoted 6 lines · Paul
Quoted 6 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 13:39
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];Quoted 4 lines
Quoted 4 lines
Stefano
Quoted 3 lines
Quoted 3 lines
Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state isforced
Quoted 38 lines
Quoted 38 lines
to
Quoted 9 lines
Quoted 9 lines
closed
Quoted 6 lines · Paul
Quoted 6 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 12:09
To: Durrant, Paul <[email protected]>; Roger Pau Monné <[email protected]>
Cc: [email protected]; [email protected];Quoted 4 lines
Quoted 4 lines
Stefano
Quoted 3 lines
Quoted 3 lines
Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state isforced
Quoted 14 lines
Quoted 14 lines
to
Quoted 13 lines
Quoted 13 lines
closed
Quoted 6 lines · Paul
Quoted 6 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 11:55
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];Stefano
Quoted 3 lines
Quoted 3 lines
Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state isQuoted 7 lines
Quoted 7 lines
forced
Quoted 6 lines
Quoted 6 lines
to
Quoted 5 lines
Quoted 5 lines
closed
Quoted 2 lines · Roger Pau Monné
Quoted 1 line · Paul Durrant
Only force state to closed in the case when the toolstack may
need
Quoted 1 line
Quoted 1 line
to
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 1 line
Quoted 1 line
clean up. This can be detected by checking whether the state in
xenstore
Quoted 3 lines
Quoted 1 line
has been set to closing prior to device removal.
I'm not sure I see the point of this, I would expect that a
failure
Quoted 2 lines
Quoted 2 lines
to
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
probe or the removal of the device would leave the xenbus state
as
Quoted 27 lines
Quoted 27 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
closed, which is consistent with the actual driver state.
Can you explain what's the benefit of leaving a device without a driver in such unknown state?
And more concerning: did you check that no frontend/backend is relying on the closed state to be visible without closing having
been
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
set before?
Blkfront doesn't seem to mind and I believe the Windows PV drivers
cope,
Quoted 1 line
Quoted 1 line
but I don't really understand the comment since this patch is
actually
Quoted 5 lines
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
removing a case where the backend transitions directly to closed.
I'm not speaking of blkfront/blkback only, but of net, tpm, scsi,
pvcall
Quoted 1 line
Quoted 1 line
etc. frontends/backends. After all you are modifying a function
common
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
to all PV driver pairs.
You are removing a state switc to "closed" in case the state was
_not_
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
"closing" before.
Yes, which AFAIK is against the intention of the generic PV protocol
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
such that it ever existed anyway.
While this might be the case we should _not_ break any guests running now. So this kind of reasoning is dangerous.
Quoted 3 lines
Quoted 3 lines
So any PV driver reacting to "closed" of the other end in case the previous state might not have been "closing" before is
at
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
risk to misbehave with your patch.
Well, they will see nothing now. If the state was not closing, it
gets
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
left alone, so the frontend shouldn't do anything. The only risk that
I
Quoted 5 lines
Quoted 5 lines
Quoted 2 lines
Quoted 2 lines
can see is that some frontend/backend pair needed a direct 4 -> 6 transition to support 'unbind' before but AFAIK nothing has ever
supported
Quoted 2 lines
Quoted 2 lines
that, and blk and net crash'n'burn if you try that on upstream as it stands. A clean unplug would always set state to 5 first, since
that's
Quoted 14 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
part of the unplug protocol.
That was my question: are you sure all current and previous guest frontends and backends are handling unplug this way?
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
Not "should handle", but "do handle".
That depends on the toolstack. IIUC the only 'supported' toolstack is
xl/libxl, which will set 'state' to 5 and 'online' to 0 to initiate an unplug.
I guess libvirt/libxl is doing the same?
Quoted 7 lines
The unplug mechansism is all in libxl AFAICT, so it should be identical.
Quoted 3 lines
At least at SUSE we still have some customers running xend based Xen installations with recent Linux or Windows guests.
Is that something the upstream code can/should support though? I'd be
surprised if xend is actually doing anything different to libxl since I've been coding the Windows PV drivers to trigger off the combined closing/online transition for as long as I can remember.
Quoted 6 lines
I'd rather not have to carry a private patch for new Linux kernel to be able to run on those hosts.
AFAIK you at Amazon have some quite old Xen installations, too. How are you handling that (assuming the customer is updating the kernel to a recent version in his guest)?
I'm not aware of any problems running with xend (but I'm not I the loop on everything). I think it is still a reasonably safe assumption that xend initiated unplug cleanly and doesn't rely on a 4 -> 6 transition.
Paul
Quoted 3 lines
Juergen
RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
In reply to Roger Pau MonnéQuoted 15 lines
-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 11:39
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Juergen Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closedQuoted 3 lines · Paul Durrant
Only force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in xenstore has been set to closing prior to device removal.
Quoted 7 lines
I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.
Can you explain what's the benefit of leaving a device without a driver in such unknown state?
If probe fails then I think it should leave the state alone. If the state is moved to closed then basically you just killed that connection to the guest (as the frontend will normally close down when it sees this change) so, if the probe failure was due to a bug in blkback or, e.g., a transient resource issue then it's game over as far as that guest goes. The ultimate goal here is PV backend re-load that is completely transparent to the guest. Modifying anything in xenstore compromises that so we need to be careful.
Paul
Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 23 lines · Paul
Quoted 15 lines
-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 11:39
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Juergen Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closedQuoted 3 lines · Paul Durrant
Only force state to closed in the case when the toolstack may need to clean up. This can be detected by checking whether the state in xenstore has been set to closing prior to device removal.
Quoted 7 lines
I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.
Can you explain what's the benefit of leaving a device without a driver in such unknown state?
Quoted 6 lines
If probe fails then I think it should leave the state alone. If the state is moved to closed then basically you just killed that connection to the guest (as the frontend will normally close down when it sees this change) so, if the probe failure was due to a bug in blkback or, e.g., a transient resource issue then it's game over as far as that guest goes.
But the connection can be restarted by switching the backend to the init state again.
Quoted 1 line
The ultimate goal here is PV backend re-load that is completely transparent to the guest. Modifying anything in xenstore compromises that so we need to be careful.
That's a fine goal, but not switching to closed state in xenbus_dev_remove seems wrong, as you have actually left the frontend without a matching backend and with the state not set to closed.
Ie: that would be fine if you explicitly state this is some kind of internal blkback reload, but not for the general case where blkback has been unbound. I think we need someway to difference a blkback reload vs a unbound.
Thanks, Roger.
RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 45 lines
-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 12:26
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Juergen Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closedQuoted 5 lines · Paul
Quoted 5 lines
-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 11:39
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];Juergen
Quoted 3 lines
Quoted 3 lines
Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state isforced to
Quoted 4 lines
Quoted 4 lines
closed
Quoted 1 line · Paul Durrant
Only force state to closed in the case when the toolstack may need
to
Quoted 1 line
Quoted 1 line
Quoted 1 line
clean up. This can be detected by checking whether the state in
xenstore
Quoted 10 lines
Quoted 9 lines
Quoted 1 line
has been set to closing prior to device removal.
I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.
Can you explain what's the benefit of leaving a device without a driver in such unknown state?
Quoted 6 lines
If probe fails then I think it should leave the state alone. If the state is moved to closed then basically you just killed that connection to the guest (as the frontend will normally close down when it sees this change) so, if the probe failure was due to a bug in blkback or, e.g., a transient resource issue then it's game over as far as that guest goes.
Quoted 2 lines
But the connection can be restarted by switching the backend to the init state again.
Too late. The frontend saw closed and you already lost.
Quoted 9 lines
Quoted 2 lines
The ultimate goal here is PV backend re-load that is completely
transparent to the guest. Modifying anything in xenstore compromises that so we need to be careful.
That's a fine goal, but not switching to closed state in
xenbus_dev_removeseems wrong, as you have actually left the frontend without a matching backend and with the state not set to closed.
Why is this a problem? With this series fully applied a (block) backend can come and go without needing to change the state. Relying on guests to DTRT is not a sustainable option for a cloud deployment.
Quoted 5 lines
Ie: that would be fine if you explicitly state this is some kind of internal blkback reload, but not for the general case where blkback has been unbound. I think we need someway to difference a blkback reload vs a unbound.
Why do we need that though? Why is it advantageous for a backend to go to closed. No PV backends cope with an unbind as-is, and a toolstack initiated unplug will always set state to 5 anyway. So TBH any state transition done directly in the xenbus code looks wrong to me anyway (but appears to be a necessary evil to keep the toolstack working in the event it spawns a backend where there is actually to driver present, or it doesn't come online).
Paul
Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 62 lines · Paul
Quoted 45 lines
-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 12:26
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Juergen Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closedQuoted 5 lines · Paul
Quoted 5 lines
-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 11:39
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];Juergen
Quoted 3 lines
Quoted 3 lines
Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state isforced to
Quoted 4 lines
Quoted 4 lines
closed
Quoted 1 line · Paul Durrant
Only force state to closed in the case when the toolstack may need
to
Quoted 1 line
Quoted 1 line
Quoted 1 line
clean up. This can be detected by checking whether the state in
xenstore
Quoted 10 lines
Quoted 9 lines
Quoted 1 line
has been set to closing prior to device removal.
I'm not sure I see the point of this, I would expect that a failure to probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.
Can you explain what's the benefit of leaving a device without a driver in such unknown state?
Quoted 6 lines
If probe fails then I think it should leave the state alone. If the state is moved to closed then basically you just killed that connection to the guest (as the frontend will normally close down when it sees this change) so, if the probe failure was due to a bug in blkback or, e.g., a transient resource issue then it's game over as far as that guest goes.
Quoted 2 lines
But the connection can be restarted by switching the backend to the init state again.
Too late. The frontend saw closed and you already lost.
Quoted 9 lines
Quoted 2 lines
The ultimate goal here is PV backend re-load that is completely
transparent to the guest. Modifying anything in xenstore compromises that so we need to be careful.
That's a fine goal, but not switching to closed state in
xenbus_dev_removeseems wrong, as you have actually left the frontend without a matching backend and with the state not set to closed.Why is this a problem? With this series fully applied a (block) backend can come and go without needing to change the state. Relying on guests to DTRT is not a sustainable option for a cloud deployment.
Quoted 7 lines
Quoted 5 lines
Ie: that would be fine if you explicitly state this is some kind of internal blkback reload, but not for the general case where blkback has been unbound. I think we need someway to difference a blkback reload vs a unbound.
Why do we need that though? Why is it advantageous for a backend to go to closed. No PV backends cope with an unbind as-is, and a toolstack initiated unplug will always set state to 5 anyway. So TBH any state transition done directly in the xenbus code looks wrong to me anyway (but appears to be a necessary evil to keep the toolstack working in the event it spawns a backend where there is actually to driver present, or it doesn't come online).
IMO the normal flow for unbind would be to attempt to close open connections and then remove the driver: leaving frontends connected without any attached backends is not correct, and will just block the guest frontend until requests start timing out.
I can see the reasoning for doing that for the purpose of updating a blkback module without guests noticing, but I would prefer that leaving connections open was an option that could be given when unbinding (or maybe a driver option in sysfs?), so that the default behaviour would be to try to close everything when unbinding if possible.
Thanks, Roger.
RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 100 lines
-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 14:29
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Juergen Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closedQuoted 5 lines · Paul
Quoted 5 lines
-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 12:26
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];Juergen
Quoted 3 lines
Quoted 3 lines
Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state isforced to
Quoted 10 lines
Quoted 10 lines
closed
Quoted 5 lines · Paul
Quoted 5 lines
-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 11:39
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];Juergen
Quoted 1 line
Quoted 1 line
Gross <[email protected]>; Stefano Stabellini
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state isforced to
Quoted 4 lines
Quoted 4 lines
closed
Quoted 1 line · Paul Durrant
Only force state to closed in the case when the toolstack may
need
Quoted 6 lines
Quoted 6 lines
to
Quoted 1 line
Quoted 1 line
Quoted 1 line
clean up. This can be detected by checking whether the state in
xenstore
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
has been set to closing prior to device removal.
I'm not sure I see the point of this, I would expect that a
failure to
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 6 lines
probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.
Can you explain what's the benefit of leaving a device without a driver in such unknown state?
If probe fails then I think it should leave the state alone. If the state is moved to closed then basically you just killed that connection to the guest (as the frontend will normally close down when it sees this change) so, if the probe failure was due to a bug in blkback or, e.g., a transient resource issue then it's game over as far as that guest goes.
Quoted 8 lines
Quoted 2 lines
But the connection can be restarted by switching the backend to the init state again.
Too late. The frontend saw closed and you already lost.
Quoted 3 lines
Quoted 2 lines
The ultimate goal here is PV backend re-load that is completely
transparent to the guest. Modifying anything in xenstore compromises
that
Quoted 8 lines
Quoted 6 lines
so we need to be careful.
That's a fine goal, but not switching to closed state in
xenbus_dev_removeseems wrong, as you have actually left the frontend without a matching backend and with the state not set to closed.Why is this a problem? With this series fully applied a (block) backend
can come and go without needing to change the state. Relying on guests to DTRT is not a sustainable option for a cloud deployment.
Quoted 8 lines
Quoted 6 lines
Ie: that would be fine if you explicitly state this is some kind of internal blkback reload, but not for the general case where blkback has been unbound. I think we need someway to difference a blkback reload vs a unbound.
Why do we need that though? Why is it advantageous for a backend to go
to closed. No PV backends cope with an unbind as-is, and a toolstack initiated unplug will always set state to 5 anyway. So TBH any state transition done directly in the xenbus code looks wrong to me anyway (but appears to be a necessary evil to keep the toolstack working in the event it spawns a backend where there is actually to driver present, or it doesn't come online).
IMO the normal flow for unbind would be to attempt to close open connections and then remove the driver: leaving frontends connected without any attached backends is not correct, and will just block the guest frontend until requests start timing out.
Quoted 6 lines
I can see the reasoning for doing that for the purpose of updating a blkback module without guests noticing, but I would prefer that leaving connections open was an option that could be given when unbinding (or maybe a driver option in sysfs?), so that the default behaviour would be to try to close everything when unbinding if possible.
Well unbind is pretty useless now IMO since bind doesn't work, and a transition straight to closed is just plain wrong anyway. But, we could have a flag that the backend driver sets to say that it supports transparent re-bind that gates this code. Would that make you feel more comfortable?
If you want unbind to actually do a proper unplug then that's extra work and not really something I want to tackle (and re-bind would still need to be toolstack initiated as something would have to re-create the xenstore area).
Paul
Quoted 2 lines
Thanks, Roger.
Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 100 lines · Paul
Quoted 100 lines
-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 14:29
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Juergen Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closedQuoted 5 lines · Paul
Quoted 5 lines
-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 12:26
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];Juergen
Quoted 3 lines
Quoted 3 lines
Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state isforced to
Quoted 10 lines
Quoted 10 lines
closed
Quoted 5 lines · Paul
Quoted 5 lines
-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 11:39
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected];Juergen
Quoted 1 line
Quoted 1 line
Gross <[email protected]>; Stefano Stabellini
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Quoted 2 lines
Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state isforced to
Quoted 4 lines
Quoted 4 lines
closed
Quoted 1 line · Paul Durrant
Only force state to closed in the case when the toolstack may
need
Quoted 6 lines
Quoted 6 lines
to
Quoted 1 line
Quoted 1 line
Quoted 1 line
clean up. This can be detected by checking whether the state in
xenstore
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
has been set to closing prior to device removal.
I'm not sure I see the point of this, I would expect that a
failure to
Quoted 14 lines
Quoted 14 lines
Quoted 13 lines
Quoted 6 lines
probe or the removal of the device would leave the xenbus state as closed, which is consistent with the actual driver state.
Can you explain what's the benefit of leaving a device without a driver in such unknown state?
If probe fails then I think it should leave the state alone. If the state is moved to closed then basically you just killed that connection to the guest (as the frontend will normally close down when it sees this change) so, if the probe failure was due to a bug in blkback or, e.g., a transient resource issue then it's game over as far as that guest goes.
Quoted 8 lines
Quoted 2 lines
But the connection can be restarted by switching the backend to the init state again.
Too late. The frontend saw closed and you already lost.
Quoted 3 lines
Quoted 2 lines
The ultimate goal here is PV backend re-load that is completely
transparent to the guest. Modifying anything in xenstore compromises
that
Quoted 8 lines
Quoted 6 lines
so we need to be careful.
That's a fine goal, but not switching to closed state in
xenbus_dev_removeseems wrong, as you have actually left the frontend without a matching backend and with the state not set to closed.Why is this a problem? With this series fully applied a (block) backend
can come and go without needing to change the state. Relying on guests to DTRT is not a sustainable option for a cloud deployment.
Quoted 8 lines
Quoted 6 lines
Ie: that would be fine if you explicitly state this is some kind of internal blkback reload, but not for the general case where blkback has been unbound. I think we need someway to difference a blkback reload vs a unbound.
Why do we need that though? Why is it advantageous for a backend to go
to closed. No PV backends cope with an unbind as-is, and a toolstack initiated unplug will always set state to 5 anyway. So TBH any state transition done directly in the xenbus code looks wrong to me anyway (but appears to be a necessary evil to keep the toolstack working in the event it spawns a backend where there is actually to driver present, or it doesn't come online).
IMO the normal flow for unbind would be to attempt to close open connections and then remove the driver: leaving frontends connected without any attached backends is not correct, and will just block the guest frontend until requests start timing out.
Quoted 8 lines
Quoted 6 lines
I can see the reasoning for doing that for the purpose of updating a blkback module without guests noticing, but I would prefer that leaving connections open was an option that could be given when unbinding (or maybe a driver option in sysfs?), so that the default behaviour would be to try to close everything when unbinding if possible.
Well unbind is pretty useless now IMO since bind doesn't work, and a transition straight to closed is just plain wrong anyway.
Why do you claim that a straight transition into the closed state is wrong?
I don't see any such mention in blkif.h, which also doesn't contain any guidelines regarding closing state transitions, so unless otherwise stated somewhere else transitions into closed can happen from any state IMO.
Quoted 1 line
But, we could have a flag that the backend driver sets to say that it supports transparent re-bind that gates this code. Would that make you feel more comfortable?
Having an option to leave state untouched when unbinding would be fine for me, otherwise state should be set to closed when unbinding. I don't think there's anything else that needs to be done in this regard, the cleanup should be exactly the same the only difference being the setting of all the active backends to closed state.
Quoted 1 line
If you want unbind to actually do a proper unplug then that's extra work and not really something I want to tackle (and re-bind would still need to be toolstack initiated as something would have to re-create the xenstore area).
Why do you say the xenstore area would need to be recreated?
Setting state to closed shouldn't cause any cleanup of the xenstore area, as that should already happen for example when using pvgrub since in that case grub itself disconnects and already causes a transition to closed and a re-attachment afterwards by the guest kernel.
Thanks, Roger.
RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 1 line
-----Original Message-----
[snip]
Quoted 6 lines
Quoted 2 lines
Well unbind is pretty useless now IMO since bind doesn't work, and a
transition straight to closed is just plain wrong anyway.
Why do you claim that a straight transition into the closed state is wrong?
It's badly documented, I agree, but have a look at https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/net/xen-netback/xenbus.c#n480. Connected -> Closed is not a valid transition, and I don't think it was ever intended to be.
Quoted 6 lines
I don't see any such mention in
blkif.h, which also doesn't contain any guidelines regarding closing state transitions, so unless otherwise stated somewhere else transitions into closed can happen from any state IMO.
They can, but it is even more poorly documented what should be done in this case.
Quoted 10 lines
Quoted 1 line
But, we could have a flag that the backend driver sets to say that it
supports transparent re-bind that gates this code. Would that make you feel more comfortable?
Having an option to leave state untouched when unbinding would be fine for me, otherwise state should be set to closed when unbinding. I don't think there's anything else that needs to be done in this regard, the cleanup should be exactly the same the only difference being the setting of all the active backends to closed state.
Ok, I'll add such a flag and define it for blkback only, in patch #4 i.e. when it actually gains the ability to rebind.
Quoted 5 lines
Quoted 1 line
If you want unbind to actually do a proper unplug then that's extra work
and not really something I want to tackle (and re-bind would still need to be toolstack initiated as something would have to re-create the xenstore area).
Quoted 8 lines
Why do you say the xenstore area would need to be recreated?
Setting state to closed shouldn't cause any cleanup of the xenstore area, as that should already happen for example when using pvgrub since in that case grub itself disconnects and already causes a transition to closed and a re-attachment afterwards by the guest kernel.
For some reason, when I originally tested, the xenstore area disappeared. I checked again and it did not this time. I just ended up with a frontend stuck in state 5 (because it is the system disk and won't go offline) trying to talk to a non-existent backend. Upon re-bind the backend goes into state 5 (because it sees the 5 in the frontend) and leaves the guest wedged.
Paul
Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 7 lines · Paul
Quoted 7 lines
Quoted 1 line
If you want unbind to actually do a proper unplug then that's extra work
and not really something I want to tackle (and re-bind would still need to be toolstack initiated as something would have to re-create the xenstore area).
Why do you say the xenstore area would need to be recreated?
Quoted 8 lines
Quoted 6 lines
Setting state to closed shouldn't cause any cleanup of the xenstore area, as that should already happen for example when using pvgrub since in that case grub itself disconnects and already causes a transition to closed and a re-attachment afterwards by the guest kernel.
For some reason, when I originally tested, the xenstore area disappeared. I checked again and it did not this time. I just ended up with a frontend stuck in state 5 (because it is the system disk and won't go offline) trying to talk to a non-existent backend. Upon re-bind the backend goes into state 5 (because it sees the 5 in the frontend) and leaves the guest wedged.
Likely blkfront should go back to init state, but anyway, that's not something that needs fixing as part of this series.
Thanks, Roger.
RE: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closed
Quoted 29 lines
-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 17:18
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Juergen Gross <[email protected]>; Stefano Stabellini <[email protected]>; Boris Ostrovsky <[email protected]>
Subject: Re: [Xen-devel] [PATCH 2/4] xenbus: limit when state is forced to closedQuoted 1 line · Paul
Quoted 1 line
Quoted 1 line
If you want unbind to actually do a proper unplug then that's extra
work
Quoted 1 line
Quoted 1 line
and not really something I want to tackle (and re-bind would still
need to
Quoted 1 line
Quoted 1 line
be toolstack initiated as something would have to re-create the
xenstore
Quoted 4 lines
Quoted 4 lines
area).
Why do you say the xenstore area would need to be recreated?
Quoted 8 lines
Quoted 6 lines
Setting state to closed shouldn't cause any cleanup of the xenstore area, as that should already happen for example when using pvgrub since in that case grub itself disconnects and already causes a transition to closed and a re-attachment afterwards by the guest kernel.
For some reason, when I originally tested, the xenstore area
Quoted 9 lines
disappeared. I checked again and it did not this time. I just ended up with a frontend stuck in state 5 (because it is the system disk and won't go offline) trying to talk to a non-existent backend. Upon re-bind the backend goes into state 5 (because it sees the 5 in the frontend) and leaves the guest wedged.
Likely blkfront should go back to init state, but anyway, that's not something that needs fixing as part of this series.
Ok, cool.
I am wondering though whether we ought to suppress bind/unbind for drivers that don't whitelist themselves (through the new xenbus_driver flag that I'll add). It's somewhat misleading that the nodes are there but don't necessarily work.
Paul
PATCH 3/4 xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH
In reply to Paul DurrantCurrently these macros will skip over any requests/responses that are added to the shared ring whilst it is detached. This, in general, is not a desirable semantic since most frontend implementations will eventually block waiting for a response which would either never appear or never be processed.
NOTE: These macros are currently unused. BACK_RING_ATTACH(), however, will
be used in a subsequent patch.
---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Stefano Stabellini <[email protected]>
--- include/xen/interface/io/ring.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
include/xen/interface/io/ring.h +2 -2
@@ -143,14 +143,14 @@ struct __name##_back_ring { \#define FRONT_RING_ATTACH (_r, _s, __size) do { \ (_r)->sring = (_s); \ (_r)->req_prod_pvt = (_s)->req_prod; \ - (_r)->rsp_cons = (_s)->rsp_prod; \ + (_r)->rsp_cons = (_s)->req_prod; \ (_r)->nr_ents = __RING_SIZE(_s, __size); \ } while (0)#define BACK_RING_ATTACH (_r, _s, __size) do { \ (_r)->sring = (_s); \ (_r)->rsp_prod_pvt = (_s)->rsp_prod; \ - (_r)->req_cons = (_s)->req_prod; \ + (_r)->req_cons = (_s)->rsp_prod; \ (_r)->nr_ents = __RING_SIZE(_s, __size); \ } while (0)
Signature · 1 line
2.20.1
Re: [Xen-devel] [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH
Quoted 10 lines · Paul Durrant
Currently these macros will skip over any requests/responses that are added to the shared ring whilst it is detached. This, in general, is not a desirable semantic since most frontend implementations will eventually block waiting for a response which would either never appear or never be processed.
NOTE: These macros are currently unused.
BACK_RING_ATTACH(), however, will
be used in a subsequent patch.Signed-off-by Paul Durrant <[email protected]>
Those headers come from Xen, and should be modified in Xen first and then imported into Linux IMO.
Thanks, Roger.
Jürgen Groß
maintainerRe: [Xen-devel] [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH
Quoted 7 lines · Roger Pau Monné
Quoted 6 lines · Paul Durrant
Currently these macros will skip over any requests/responses that are added to the shared ring whilst it is detached. This, in general, is not a desirable semantic since most frontend implementations will eventually block waiting for a response which would either never appear or never be processed.
Quoted 7 lines
Quoted 4 lines
NOTE: These macros are currently unused.
BACK_RING_ATTACH(), however, will
be used in a subsequent patch.Signed-off-by Paul Durrant <[email protected]>Those headers come from Xen, and should be modified in Xen first and then imported into Linux IMO.
In theory, yes. But the Xen variant doesn't contain the ATTACH macros.
Juergen
RE: [Xen-devel] [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH
Quoted 25 lines
-----Original Message-----
From: Xen-devel <[email protected]> On Behalf Of Jürgen Groß
Sent: 09 December 2019 11:52
To: Roger Pau Monné <[email protected]>; Durrant, Paul <[email protected]>
Cc: [email protected]; Boris Ostrovsky <[email protected]>; Stefano Stabellini <[email protected]>; [email protected]
Subject: Re: [Xen-devel] [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACHQuoted 3 lines · Roger Pau Monné
Quoted 2 lines · Paul Durrant
Currently these macros will skip over any requests/responses that are added to the shared ring whilst it is detached. This, in general, is
not
Quoted 1 line
Quoted 1 line
a desirable semantic since most frontend implementations will
eventually
Quoted 1 line
Quoted 1 line
block waiting for a response which would either never appear or never
be
Quoted 3 lines
Quoted 3 lines
processed.
NOTE: These macros are currently unused.
BACK_RING_ATTACH(), however,will
Quoted 9 lines
Quoted 6 lines
Quoted 3 lines
be used in a subsequent patch.
Signed-off-by Paul Durrant <[email protected]>Those headers come from Xen, and should be modified in Xen first and then imported into Linux IMO.
In theory, yes. But the Xen variant doesn't contain the ATTACH macros.
OOI do we have a policy about this? Re-importing headers into Linux wholesale is always slightly painful because of interdependencies and style checking issues.
Paul
Quoted 7 lines
Juergen
_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel
Jürgen Groß
maintainerRe: [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH
In reply to Paul DurrantQuoted 28 lines · Paul Durrant
Currently these macros will skip over any requests/responses that are added to the shared ring whilst it is detached. This, in general, is not a desirable semantic since most frontend implementations will eventually block waiting for a response which would either never appear or never be processed.
NOTE: These macros are currently unused.
BACK_RING_ATTACH(), however, will
be used in a subsequent patch.Signed-off-by Paul Durrant <[email protected]>---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Stefano Stabellini <[email protected]>--- include/xen/interface/io/ring.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)include/xen/interface/io/ring.h +1
-1@@ -143,14 +143,14 @@ struct __name##_back_ring { \#define FRONT_RING_ATTACH (_r, _s, __size) do { \ (_r)->sring = (_s); \ (_r)->req_prod_pvt = (_s)->req_prod; \ - (_r)->rsp_cons = (_s)->rsp_prod; \ + (_r)->rsp_cons = (_s)->req_prod; \
include/xen/interface/io/ring.h struct __name##_back_ring { \ · line 155
Quoted 10 lines
include/xen/interface/io/ring.h +1
-1(_r)->nr_ents =__RING_SIZE (_s, __size); \ } while (0)#define BACK_RING_ATTACH (_r, _s, __size) do { \ (_r)->sring = (_s); \ (_r)->rsp_prod_pvt = (_s)->rsp_prod; \ - (_r)->req_cons = (_s)->req_prod; \ + (_r)->req_cons = (_s)->rsp_prod; \ (_r)->nr_ents = __RING_SIZE(_s, __size); \ } while (0)
Lets look at all possible scenarios where BACK_RING_ATTACH() might happen:
Initially (after [FRONT|BACK]_RING_INIT(), leaving _pvt away):
req_prod=0, rsp_cons=0, rsp_prod=0, req_cons=0
Using BACK_RING_ATTACH() is fine (no change)
Request queued:
req_prod=1, rsp_cons=0, rsp_prod=0, req_cons=0
Using BACK_RING_ATTACH() is fine (no change)
and taken by backend:
req_prod=1, rsp_cons=0, rsp_prod=0, req_cons=1
Using BACK_RING_ATTACH() is resetting req_cons to 0, will result in redoing request (for blk this is fine, other devices like SCSI tapes will have issues with that). One possible solution would be to ensure all taken requests are either stopped or the response is queued already.
Response queued:
req_prod=1, rsp_cons=0, rsp_prod=1, req_cons=1
Using BACK_RING_ATTACH() is fine (no change)
Response taken:
req_prod=1, rsp_cons=1, rsp_prod=1, req_cons=1
Using BACK_RING_ATTACH() is fine (no change)
In general I believe the [FRONT|BACK]_RING_ATTACH() macros are not fine to be used in the current state, as the *_pvt fields normally not accessible by the other end are initialized using the (possibly untrusted) values from the shared ring. There needs at least to be a test for the values to be sane, and your change should not result in the same value to be read twice, as it could have changed in between.
As this is an error which can happen in other OS's, too, I'd recommend to add the adapted macros (plus a comment regarding the possible problem noted above for special devices like tapes) to the Xen variant of ring.h.
Juergen
RE: [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH
Quoted 64 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 13:55
To: Durrant, Paul <[email protected]>; [email protected]; [email protected]
Cc: Boris Ostrovsky <[email protected]>; Stefano Stabellini <[email protected]>
Subject: Re: [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACHQuoted 7 lines · Paul Durrant
Currently these macros will skip over any requests/responses that are added to the shared ring whilst it is detached. This, in general, is not a desirable semantic since most frontend implementations will eventually block waiting for a response which would either never appear or never be processed.
NOTE: These macros are currently unused.
BACK_RING_ATTACH(), however,will
Quoted 7 lines
be used in a subsequent patch.
Signed-off-by Paul Durrant <[email protected]>---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Stefano Stabellini <[email protected]>Review
Quoted 5 lines
--- include/xen/interface/io/ring.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/include/xen/interface/io/ring.hinclude/xen/interface/io/ring.h struct __name##_back_ring {
Quoted 4 lines
index
3f40501fc60b..405adfed87e6100644include/xen/interface/io/ring.h +0
-0@@ -143,14 +143,14 @@ struct __name##_back_ring {\Quoted 5 lines
include/xen/interface/io/ring.h +1
-1#define FRONT_RING_ATTACH (_r, _s, __size) do { \ (_r)->sring = (_s); \ (_r)->req_prod_pvt = (_s)->req_prod; \ - (_r)->rsp_cons = (_s)->rsp_prod; \ + (_r)->rsp_cons = (_s)->req_prod; \include/xen/interface/io/ring.h struct __name##_back_ring { \ · line 155
Quoted 10 lines
include/xen/interface/io/ring.h +1
-1(_r)->nr_ents =__RING_SIZE (_s, __size); \ } while (0)#define BACK_RING_ATTACH (_r, _s, __size) do { \ (_r)->sring = (_s); \ (_r)->rsp_prod_pvt = (_s)->rsp_prod; \ - (_r)->req_cons = (_s)->req_prod; \ + (_r)->req_cons = (_s)->rsp_prod; \ (_r)->nr_ents = __RING_SIZE(_s, __size); \ } while (0)Lets look at all possible scenarios where
BACK_RING_ATTACH()might happen:Initially (after [FRONT|BACK]_RING_INIT(), leaving
_pvtaway):
req_prod=0, rsp_cons=0, rsp_prod=0, req_cons=0
UsingBACK_RING_ATTACH()is fine (no change)Request queued:
req_prod=1, rsp_cons=0, rsp_prod=0, req_cons=0
UsingBACK_RING_ATTACH()is fine (no change)
Quoted 7 lines
and taken by backend:
req_prod=1, rsp_cons=0, rsp_prod=0, req_cons=1
UsingBACK_RING_ATTACH()is resettingreq_consto 0, will result in redoing request (for blk this is fine, other devices like SCSI tapes will have issues with that). One possible solution would be to ensure all taken requests are either stopped or the response is queued already.
Yes, it is the assumption that a backend will drain and complete any requests it is handling, but it will not deal with new ones being posted by the frontend. This does appear to be the case for blkback.
Quoted 9 lines
Response queued:
req_prod=1, rsp_cons=0, rsp_prod=1, req_cons=1
UsingBACK_RING_ATTACH()is fine (no change)Response taken:
req_prod=1, rsp_cons=1, rsp_prod=1, req_cons=1
UsingBACK_RING_ATTACH()is fine (no change)
Quoted 6 lines
In general I believe the [FRONT|BACK]_RING_ATTACH() macros are not fine to be used in the current state, as the
*_pvtfields normally not accessible by the other end are initialized using the (possibly untrusted) values from the shared ring. There needs at least to be a test for the values to be sane, and your change should not result in the same value to be read twice, as it could have changed in between.
What test would you apply to sanitize the value of the pvt pointer? Another option would be to have a backend write its pvt value into the xenstore backend area when the ring is unmapped, so that a new instance definitely resumes where the old one left off. The value of rsp_prod could, of course, be overwritten by the guest at any time and so there's little point in attempting sanitize it.
Quoted 6 lines
As this is an error which can happen in other OS's, too, I'd recommend to add the adapted macros (plus a comment regarding the possible problem noted above for special devices like tapes) to the Xen variant of
ring.h.
I can certainly send a patch to Xen once we agree on the final definition.
Paul
Quoted 2 lines
Juergen
Jürgen Groß
maintainerRe: [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACH
Quoted 83 lines · Paul
Quoted 64 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 13:55
To: Durrant, Paul <[email protected]>; [email protected]; [email protected]
Cc: Boris Ostrovsky <[email protected]>; Stefano Stabellini <[email protected]>
Subject: Re: [PATCH 3/4] xen/interface: don't discard pending work in FRONT/BACK_RING_ATTACHQuoted 7 lines · Paul Durrant
Currently these macros will skip over any requests/responses that are added to the shared ring whilst it is detached. This, in general, is not a desirable semantic since most frontend implementations will eventually block waiting for a response which would either never appear or never be processed.
NOTE: These macros are currently unused.
BACK_RING_ATTACH(), however,will
Quoted 7 lines
be used in a subsequent patch.
Signed-off-by Paul Durrant <[email protected]>---
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Stefano Stabellini <[email protected]>Review
Quoted 5 lines
--- include/xen/interface/io/ring.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/include/xen/interface/io/ring.hinclude/xen/interface/io/ring.h struct __name##_back_ring {
Quoted 4 lines
index
3f40501fc60b..405adfed87e6100644include/xen/interface/io/ring.h +0
-0@@ -143,14 +143,14 @@ struct __name##_back_ring {\Quoted 5 lines
include/xen/interface/io/ring.h +1
-1#define FRONT_RING_ATTACH (_r, _s, __size) do { \ (_r)->sring = (_s); \ (_r)->req_prod_pvt = (_s)->req_prod; \ - (_r)->rsp_cons = (_s)->rsp_prod; \ + (_r)->rsp_cons = (_s)->req_prod; \include/xen/interface/io/ring.h struct __name##_back_ring { \ · line 155
Quoted 10 lines
include/xen/interface/io/ring.h +1
-1(_r)->nr_ents =__RING_SIZE (_s, __size); \ } while (0)#define BACK_RING_ATTACH (_r, _s, __size) do { \ (_r)->sring = (_s); \ (_r)->rsp_prod_pvt = (_s)->rsp_prod; \ - (_r)->req_cons = (_s)->req_prod; \ + (_r)->req_cons = (_s)->rsp_prod; \ (_r)->nr_ents = __RING_SIZE(_s, __size); \ } while (0)Lets look at all possible scenarios where
BACK_RING_ATTACH()might happen:Initially (after [FRONT|BACK]_RING_INIT(), leaving
_pvtaway):
req_prod=0, rsp_cons=0, rsp_prod=0, req_cons=0
UsingBACK_RING_ATTACH()is fine (no change)Request queued:
req_prod=1, rsp_cons=0, rsp_prod=0, req_cons=0
UsingBACK_RING_ATTACH()is fine (no change)Quoted 7 lines
and taken by backend:
req_prod=1, rsp_cons=0, rsp_prod=0, req_cons=1
UsingBACK_RING_ATTACH()is resettingreq_consto 0, will result in redoing request (for blk this is fine, other devices like SCSI tapes will have issues with that). One possible solution would be to ensure all taken requests are either stopped or the response is queued already.Yes, it is the assumption that a backend will drain and complete any requests it is handling, but it will not deal with new ones being posted by the frontend. This does appear to be the case for blkback.
Quoted 9 lines
Response queued:
req_prod=1, rsp_cons=0, rsp_prod=1, req_cons=1
UsingBACK_RING_ATTACH()is fine (no change)Response taken:
req_prod=1, rsp_cons=1, rsp_prod=1, req_cons=1
UsingBACK_RING_ATTACH()is fine (no change)
Quoted 8 lines
Quoted 6 lines
In general I believe the [FRONT|BACK]_RING_ATTACH() macros are not fine to be used in the current state, as the
*_pvtfields normally not accessible by the other end are initialized using the (possibly untrusted) values from the shared ring. There needs at least to be a test for the values to be sane, and your change should not result in the same value to be read twice, as it could have changed in between.What test would you apply to sanitize the value of the pvt pointer?
For the BACK_RING_ATTACH() case rsp_prod_pvt should not be between req_prod and req_cons, and req_cons - rsp_prod_pvt should be <= ring size IMO.
Quoted 1 line
Another option would be to have a backend write its pvt value into the xenstore backend area when the ring is unmapped, so that a new instance definitely resumes where the old one left off. The value of
rsp_prodcould, of course, be overwritten by the guest at any time and so there's little point in attempting sanitize it.
I don't think this would be necessary. With above validation in place all the guest could do would be to shoot itself in the foot.
Juergen
PATCH 4/4 xen-blkback: support dynamic unbind/bind
In reply to Paul DurrantBy simply re-attaching to shared rings during connect_ring() rather than assuming they are freshly allocated (i.e assuming the counters are zero) it is possible for vbd instances to be unbound and re-bound from and to (respectively) a running guest.
This has been tested by running:
while true; do dd if=/dev/urandom of=test.img bs=1M count=1024; done
in a PV guest whilst running:
while true;
do echo vbd-$DOMID-$VBD >unbind;
echo unbound;
sleep 5;
echo vbd-$DOMID-$VBD >bind;
echo bound;
sleep 3;
done
in dom0 from /sys/bus/xen-backend/drivers/vbd to continuously unbind and re-bind its system disk image.
This is a highly useful feature for a backend module as it allows it to be unloaded and re-loaded (i.e. updated) without requiring domUs to be halted. This was also tested by running:
while true;
do echo vbd-$DOMID-$VBD >unbind;
echo unbound;
sleep 5;
rmmod xen-blkback;
echo unloaded;
sleep 1;
modprobe xen-blkback;
echo bound;
cd $(pwd);
sleep 3;
done
in dom0 whilst running the same loop as above in the (single) PV guest.
Some (less stressful) testing has also been done using a Windows HVM guest with the latest 9.0 PV drivers installed.
---
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
Cc: Jens Axboe <[email protected]>
Cc: Boris Ostrovsky <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Stefano Stabellini <[email protected]>
--- drivers/block/xen-blkback/xenbus.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
drivers/block/xen-blkback/xenbus.c +6 -6
@@ -196,24 +196,24 @@ static int xen_blkif_map(struct xen_blkif_ring *ring, grant_ref_t *gref, {struct blkif_sring *sring; sring = (struct blkif_sring *)ring->blk_ring; -BACK_RING_INIT (&ring->blk_rings.native, sring, -XEN_PAGE_SIZE * nr_grefs); +BACK_RING_ATTACH (&ring->blk_rings.native, sring, +XEN_PAGE_SIZE * nr_grefs);break ; }case BLKIF_PROTOCOL_X86_32 : {struct blkif_x86_32_sring *sring_x86_32; sring_x86_32 = (struct blkif_x86_32_sring *)ring->blk_ring; -BACK_RING_INIT (&ring->blk_rings.x86_32, sring_x86_32, -XEN_PAGE_SIZE * nr_grefs); +BACK_RING_ATTACH (&ring->blk_rings.x86_32, sring_x86_32, +XEN_PAGE_SIZE * nr_grefs);break ; }case BLKIF_PROTOCOL_X86_64 : {struct blkif_x86_64_sring *sring_x86_64; sring_x86_64 = (struct blkif_x86_64_sring *)ring->blk_ring; -BACK_RING_INIT (&ring->blk_rings.x86_64, sring_x86_64, -XEN_PAGE_SIZE * nr_grefs); +BACK_RING_ATTACH (&ring->blk_rings.x86_64, sring_x86_64, +XEN_PAGE_SIZE * nr_grefs);break ; }default :
Signature · 1 line
2.20.1
Re: [PATCH 4/4] xen-blkback: support dynamic unbind/bind
Quoted 11 lines · Paul Durrant
By simply re-attaching to shared rings during
connect_ring()rather than assuming they are freshly allocated (i.e assuming the counters are zero) it is possible for vbd instances to be unbound and re-bound from and to (respectively) a running guest.This has been tested by running:
while true; do dd if=/dev/urandom of=test.img bs=1M count=1024; done
in a PV guest whilst running:
Quoted 8 lines
while true;
do echo vbd-$DOMID-$VBD >unbind;
echo unbound;
sleep 5;
echo vbd-$DOMID-$VBD >bind;
echo bound;
sleep 3;
done
So this does unbind blkback while leaving the PV interface as connected?
Thanks, Roger.
RE: [PATCH 4/4] xen-blkback: support dynamic unbind/bind
Quoted 31 lines
-----Original Message-----
From: Roger Pau Monné <[email protected]>
Sent: 09 December 2019 12:17
To: Durrant, Paul <[email protected]>
Cc: [email protected]; [email protected]; Konrad Rzeszutek Wilk <[email protected]>; Jens Axboe <[email protected]>; Boris Ostrovsky <[email protected]>; Juergen Gross <[email protected]>; Stefano Stabellini <[email protected]>
Subject: Re: [PATCH 4/4] xen-blkback: support dynamic unbind/bindQuoted 11 lines · Paul Durrant
By simply re-attaching to shared rings during
connect_ring()rather than assuming they are freshly allocated (i.e assuming the counters are zero) it is possible for vbd instances to be unbound and re-bound from and to (respectively) a running guest.This has been tested by running:
while true; do dd if=/dev/urandom of=test.img bs=1M count=1024; done
in a PV guest whilst running:
Quoted 8 lines
while true;
do echo vbd-$DOMID-$VBD >unbind;
echo unbound;
sleep 5;
echo vbd-$DOMID-$VBD >bind;
echo bound;
sleep 3;
done
Quoted 3 lines
So this does unbind blkback while leaving the PV interface as connected?
Yes, everything is left in place in the frontend. The backend detaches from the ring, closes its end of the event channels, etc. but the guest can still send requests which will get serviced when the new backend attaches.
Paul
Jürgen Groß
maintainerRe: [PATCH 4/4] xen-blkback: support dynamic unbind/bind
In reply to Paul DurrantQuoted 20 lines · Paul Durrant
By simply re-attaching to shared rings during
connect_ring()rather than assuming they are freshly allocated (i.e assuming the counters are zero) it is possible for vbd instances to be unbound and re-bound from and to (respectively) a running guest.This has been tested by running:
while true; do dd if=/dev/urandom of=test.img bs=1M count=1024; done
in a PV guest whilst running:
while true; do echo vbd-$DOMID-$VBD >unbind; echo unbound; sleep 5; echo vbd-$DOMID-$VBD >bind; echo bound; sleep 3; done
Quoted 2 lines
in dom0 from
/sys/bus/xen-backend/drivers/vbdto continuously unbind and re-bind its system disk image.
Could you do the same test with mixed reads/writes and verification of the read/written data, please? A write-only test is not _that_ convincing regarding correctness. It only proves the guest is not crashing.
I'm fine with the general approach, though.
Juergen
RE: [PATCH 4/4] xen-blkback: support dynamic unbind/bind
Quoted 31 lines
-----Original Message-----
From: Jürgen Groß <[email protected]>
Sent: 09 December 2019 13:58
To: Durrant, Paul <[email protected]>; [email protected]; [email protected]
Cc: Konrad Rzeszutek Wilk <[email protected]>; Roger Pau Monné <[email protected]>; Jens Axboe <[email protected]>; Boris Ostrovsky <[email protected]>; Stefano Stabellini <[email protected]>
Subject: Re: [PATCH 4/4] xen-blkback: support dynamic unbind/bindQuoted 20 lines · Paul Durrant
By simply re-attaching to shared rings during
connect_ring()rather than assuming they are freshly allocated (i.e assuming the counters are zero) it is possible for vbd instances to be unbound and re-bound from and to (respectively) a running guest.This has been tested by running:
while true; do dd if=/dev/urandom of=test.img bs=1M count=1024; done
in a PV guest whilst running:
while true; do echo vbd-$DOMID-$VBD >unbind; echo unbound; sleep 5; echo vbd-$DOMID-$VBD >bind; echo bound; sleep 3; done
Quoted 7 lines
Quoted 2 lines
in dom0 from
/sys/bus/xen-backend/drivers/vbdto continuously unbind and re-bind its system disk image.Could you do the same test with mixed reads/writes and verification of the read/written data, please? A write-only test is not _that_ convincing regarding correctness. It only proves the guest is not crashing.
Sure. I'll find something that will verify content.
Quoted 3 lines
I'm fine with the general approach, though.
Cool, thanks,
Paul
Quoted 2 lines
Juergen