xen-blkfront: Don't send closing notification to backend in blkfront_closing()
Subsystems (3)
PATCH 1/1 xen-blkfront: Don't send closing notification to backend in blkfront_closing()
When we do block device attach/detach test with below steps, umount hang and the guest unable to shutdown:
1. start guest with the latest kernel.
2. attach new block device by xm block-attach in Dom0
3. mount new disk in guest
4. execute xm block-detach to detach the block device in dom0 until timeout
5. try to unmount the disk in guest, umount hung. at here, any IOs to the device will hang.
Checking the code found when 'xm block-detach' set backend device's state to 'XenbusStateClosing', frontend received the notification and blkfront_closing() be called, at the moment, the disk still using by guest, so frontend refused to close. In the blkfront_closing(), frontend send a notification to backend said that the its state switched to 'Closing', when backend got the event, it will disconnect from real device, at here any IO request will be stuck, even tried to release the disk by umount.
Per our test, below patch fix this issue.
--- xen-blkfront.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/block/xen-blkfront.c +1 -1
@@ -1088,7 +1088,7 @@ blkfront_closing(struct blkfront_info *info)if (bdev->bd_openers) {xenbus_dev_error (xbdev, -EBUSY ,"Device in use; refusing to close" ); -xenbus_switch_state (xbdev, XenbusStateClosing); + xbdev->state = XenbusStateClosing; }else {xlvbd_release_gendisk (info);xenbus_frontend_closed (xbdev);
Quoted 18 lines · Joe Jin
When we do block device attach/detach test with below steps, umount hang and the guest unable to shutdown:
1. start guest with the latest kernel.
2. attach new block device by xm block-attach in Dom0
3. mount new disk in guest
4. execute xm block-detach to detach the block device in dom0 until timeout
5. try to unmount the disk in guest, umount hung. at here, any IOs to the device will hang.Checking the code found when 'xm block-detach' set backend device's state to 'XenbusStateClosing', frontend received the notification and
blkfront_closing()be called, at the moment, the disk still using by guest, so frontend refused to close. In theblkfront_closing(), frontend send a notification to backend said that the its state switched to 'Closing', when backend got the event, it will disconnect from real device, at here any IO request will be stuck, even tried to release the disk by umount.
Quoted 1 line
Per our test, below patch fix this issue.
It's worth mentioning here that the change to xbdev->state is picked up in blkif_release() when the device is closed and the disconnect happens at that point instead.
I'm wondering if we might not be better off deferring the disconnect on the backend side until the frontend enters XenbusStateClosed instead of doing it in closing.
Ian
Quoted 24 lines
Signed-off-by Joe Jin <[email protected]>Signed-off-by Annie Li <[email protected]>Reviewed-by Konrad Rzeszutek Wilk <[email protected]>Cc Jens Axboe <[email protected]>--- xen-blkfront.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/block/xen-blkfront.c +1
-1@@ -1088,7 +1088,7 @@ blkfront_closing(struct blkfront_info *info)if (bdev->bd_openers) {xenbus_dev_error (xbdev, -EBUSY ,"Device in use; refusing to close" ); -xenbus_switch_state (xbdev, XenbusStateClosing); + xbdev->state = XenbusStateClosing; }else {xlvbd_release_gendisk (info);xenbus_frontend_closed (xbdev);
Quoted 19 lines · Ian Campbell
Quoted 18 lines · Joe Jin
When we do block device attach/detach test with below steps, umount hang and the guest unable to shutdown:
1. start guest with the latest kernel.
2. attach new block device by xm block-attach in Dom0
3. mount new disk in guest
4. execute xm block-detach to detach the block device in dom0 until timeout
5. try to unmount the disk in guest, umount hung. at here, any IOs to the device will hang.Checking the code found when 'xm block-detach' set backend device's state to 'XenbusStateClosing', frontend received the notification and
blkfront_closing()be called, at the moment, the disk still using by guest, so frontend refused to close. In theblkfront_closing(), frontend send a notification to backend said that the its state switched to 'Closing', when backend got the event, it will disconnect from real device, at here any IO request will be stuck, even tried to release the disk by umount.
Quoted 5 lines
Quoted 1 line
Per our test, below patch fix this issue.
It's worth mentioning here that the change to
xbdev->stateis picked up inblkif_release()when the device is closed and the disconnect happens at that point instead.
This is right, thanks for the suggestions.
Quoted 4 lines
I'm wondering if we might not be better off deferring the disconnect on the backend side until the frontend enters XenbusStateClosed instead of doing it in closing.
Yes this fix from backend side works too, also this looks reasonable than fix in frontend.
Konrad, any advice?
Thanks,
Joe
Quoted 29 lines
Ian
Quoted 24 lines
Signed-off-by Joe Jin <[email protected]>Signed-off-by Annie Li <[email protected]>Reviewed-by Konrad Rzeszutek Wilk <[email protected]>Cc Jens Axboe <[email protected]>--- xen-blkfront.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/block/xen-blkfront.c +1
-1@@ -1088,7 +1088,7 @@ blkfront_closing(struct blkfront_info *info)if (bdev->bd_openers) {xenbus_dev_error (xbdev, -EBUSY ,"Device in use; refusing to close" ); -xenbus_switch_state (xbdev, XenbusStateClosing); + xbdev->state = XenbusStateClosing; }else {xlvbd_release_gendisk (info);xenbus_frontend_closed (xbdev);
Signature · 4 lines
Oracle <http://www.oracle.com> Joe Jin | Team Leader, Software Development | +8610.6106.5624 ORACLE | Linux and Virtualization No. 24 Zhongguancun Software Park, Haidian District | 100193 Beijing
Quoted 10 lines · Joe Jin
Quoted 6 lines · Ian Campbell
On Wed, 2011-07-13 at 01:47 +0100, Joe Jin wrote:
I'm wondering if we might not be better off deferring the disconnect on the backend side until the frontend enters XenbusStateClosed instead of doing it in closing.
Yes this fix from backend side works too, also this looks reasonable than fix in frontend.
I guess there is either a missing "more" or "less" in that sentence ;-)
The nice thing about a backend fix is that you don't need to go round propagating the fix into distros etc and updating existing deployed guests (not to mention non-Linux frontends).
I actually thought blkback had something like this already in older "classic" Xen kernels but it seems like the patch never made it out of the XCP patch queue :-(. See
http://xenbits.xen.org/hg/XCP/linux-2.6.32.pq.hg/file/tip/CA-7672-blkback-shutdown.patch
(which appears to also incorporate a toolstack directed graceful shutdown of some sort as well as deferring the device close).
Daniel, do you know of any plans to extricate that stuff from the XCP patch queue for upstream? Are there any other patches (e.g. incremental fixes) in that pq which should go along with it?
Ian.
Quoted 37 lines
Konrad, any advice?
Thanks,
JoeQuoted 29 lines
Ian
Quoted 24 lines
Signed-off-by Joe Jin <[email protected]>Signed-off-by Annie Li <[email protected]>Reviewed-by Konrad Rzeszutek Wilk <[email protected]>Cc Jens Axboe <[email protected]>--- xen-blkfront.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/block/xen-blkfront.c +1
-1@@ -1088,7 +1088,7 @@ blkfront_closing(struct blkfront_info *info)if (bdev->bd_openers) {xenbus_dev_error (xbdev, -EBUSY ,"Device in use; refusing to close" ); -xenbus_switch_state (xbdev, XenbusStateClosing); + xbdev->state = XenbusStateClosing; }else {xlvbd_release_gendisk (info);xenbus_frontend_closed (xbdev);
Quoted 9 lines · Ian Campbell
Quoted 8 lines · Joe Jin
Quoted 6 lines · Ian Campbell
On Wed, 2011-07-13 at 01:47 +0100, Joe Jin wrote:
I'm wondering if we might not be better off deferring the disconnect on the backend side until the frontend enters XenbusStateClosed instead of doing it in closing.
Quoted 8 lines
Quoted 2 lines
Yes this fix from backend side works too, also this looks reasonable than fix in frontend.
I guess there is either a missing "more" or "less" in that sentence ;-)
The nice thing about a backend fix is that you don't need to go round propagating the fix into distros etc and updating existing deployed guests (not to mention non-Linux frontends).
Indeed ;)
Quoted 11 lines
I actually thought blkback had something like this already in older "classic" Xen kernels but it seems like the patch never made it out of the XCP patch queue :-(. See
http://xenbits.xen.org/hg/XCP/linux-2.6.32.pq.hg/file/tip/CA-7672-blkback-shutdown.patch
(which appears to also incorporate a toolstack directed graceful shutdown of some sort as well as deferring the device close).Daniel, do you know of any plans to extricate that stuff from the XCP patch queue for upstream? Are there any other patches (e.g. incremental fixes) in that pq which should go along with it?
Looking forward push this patch to upstream!
Joe
_______________________________________________
stable mailing list
[email protected]
http://linux.kernel.org/mailman/listinfo/stable
Quoted 29 lines · Joe Jin
Quoted 9 lines · Ian Campbell
Quoted 8 lines · Joe Jin
Quoted 6 lines · Ian Campbell
On Wed, 2011-07-13 at 01:47 +0100, Joe Jin wrote:
I'm wondering if we might not be better off deferring the disconnect on the backend side until the frontend enters XenbusStateClosed instead of doing it in closing.
Quoted 8 lines
Quoted 2 lines
Yes this fix from backend side works too, also this looks reasonable than fix in frontend.
I guess there is either a missing "more" or "less" in that sentence ;-)
The nice thing about a backend fix is that you don't need to go round propagating the fix into distros etc and updating existing deployed guests (not to mention non-Linux frontends).
Indeed ;)
Quoted 8 lines
I actually thought blkback had something like this already in older "classic" Xen kernels but it seems like the patch never made it out of the XCP patch queue :-(. See
http://xenbits.xen.org/hg/XCP/linux-2.6.32.pq.hg/file/tip/CA-7672-blkback-shutdown.patch
(which appears to also incorporate a toolstack directed graceful shutdown of some sort as well as deferring the device close).
Quoted 5 lines
Quoted 3 lines
Daniel, do you know of any plans to extricate that stuff from the XCP patch queue for upstream? Are there any other patches (e.g. incremental fixes) in that pq which should go along with it?
Looking forward push this patch to upstream!
Not sure if Daniel is going to get to this. Joe, and chance you could take a look at it and up-port in the 3.0 kernel?