Skip to contentSkip to conversation

xen-netfront: Fix hang on device removal

PATCH In review Jason Andryuk 4 participants 13 messages Feb 28, 2018 – Apr 20, 2018

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 xen-netfront: Fix hang on device removal

A toolstack may delete the vif frontend and backend xenstore entries while xen-netfront is in the removal code path. In that case, the checks for xenbus_read_driver_state would return XenbusStateUnknown, and xennet_remove would hang indefinitely. This hang prevents system shutdown.

xennet_remove must be able to handle XenbusStateUnknown, and netback_changed must also wake up the wake_queue for that state as well.

Fixes 5b5971df3bc2 ("xen-netfront: remove warning when unloading module")
---
 drivers/net/xen-netfront.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
drivers/net/xen-netfront.c +6 -1
index 8328d395e332..3127bc8633ca 100644
@@ -2005,7 +2005,10 @@ static void netback_changed(struct xenbus_device *dev,
 	case XenbusStateInitialised:
 	case XenbusStateReconfiguring:
 	case XenbusStateReconfigured:
+		break;
+
 	case XenbusStateUnknown:
+		wake_up_all(&module_unload_q);
 		break;
 
 	case XenbusStateInitWait:
@@ -2136,7 +2139,9 @@ static int xennet_remove(struct xenbus_device *dev)
 		xenbus_switch_state(dev, XenbusStateClosing);
 		wait_event(module_unload_q,
 			   xenbus_read_driver_state(dev->otherend) ==
-			   XenbusStateClosing);
+			   XenbusStateClosing ||
+			   xenbus_read_driver_state(dev->otherend) ==
+			   XenbusStateUnknown);
 
 		xenbus_switch_state(dev, XenbusStateClosed);
 		wait_event(module_unload_q,
Signature · 1 line
2.14.3
Reviewed-by on v1 1/1
#1 lore raw
Quoted 6 lines · Jason Andryuk

A toolstack may delete the vif frontend and backend xenstore entries while xen-netfront is in the removal code path. In that case, the checks for xenbus_read_driver_state would return XenbusStateUnknown, and xennet_remove would hang indefinitely. This hang prevents system shutdown.

Quoted 7 lines

xennet_remove must be able to handle XenbusStateUnknown, and netback_changed must also wake up the wake_queue for that state as well.

Fixes 5b5971df3bc2 ("xen-netfront: remove warning when unloading module")
Reviewed-by on v1 1/1
#2 lore raw
In reply to Jason Andryuk
Quoted 6 lines · Jason Andryuk

A toolstack may delete the vif frontend and backend xenstore entries while xen-netfront is in the removal code path. In that case, the checks for xenbus_read_driver_state would return XenbusStateUnknown, and xennet_remove would hang indefinitely. This hang prevents system shutdown.

Quoted 7 lines

xennet_remove must be able to handle XenbusStateUnknown, and netback_changed must also wake up the wake_queue for that state as well.

Fixes 5b5971df3bc2 ("xen-netfront: remove warning when unloading module")

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

maintainer
#3 lore raw
In reply to Jason Andryuk
Quoted 6 lines · Jason Andryuk

A toolstack may delete the vif frontend and backend xenstore entries while xen-netfront is in the removal code path. In that case, the checks for xenbus_read_driver_state would return XenbusStateUnknown, and xennet_remove would hang indefinitely. This hang prevents system shutdown.

Quoted 7 lines

xennet_remove must be able to handle XenbusStateUnknown, and netback_changed must also wake up the wake_queue for that state as well.

Fixes 5b5971df3bc2 ("xen-netfront: remove warning when unloading module")

Committed to xen/tip for-linus-4.16a

Juergen

maintainer
#4 lore raw
In reply to Jason Andryuk
Quoted 6 lines · Jason Andryuk

A toolstack may delete the vif frontend and backend xenstore entries while xen-netfront is in the removal code path. In that case, the checks for xenbus_read_driver_state would return XenbusStateUnknown, and xennet_remove would hang indefinitely. This hang prevents system shutdown.

Quoted 7 lines

xennet_remove must be able to handle XenbusStateUnknown, and netback_changed must also wake up the wake_queue for that state as well.

Fixes 5b5971df3bc2 ("xen-netfront: remove warning when unloading module")

Committed to xen/tip for-linus-4.16a

Juergen

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#5 lore raw
In reply to Jason Andryuk

Jason Andryuk:

Quoted 10 lines

A toolstack may delete the vif frontend and backend xenstore entries while xen-netfront is in the removal code path. In that case, the checks for xenbus_read_driver_state would return XenbusStateUnknown, and xennet_remove would hang indefinitely. This hang prevents system shutdown.

xennet_remove must be able to handle XenbusStateUnknown, and netback_changed must also wake up the wake_queue for that state as well.

Fixes 5b5971df3bc2 ("xen-netfront: remove warning when unloading module")

I think this should go into stable since AFAIK the hanging network device can only be fixed by rebooting the guest. AFAICS this affects all 4.* branches since 5b5971df3bc2 got backported to them.

Upstream commit c2d2e6738a209f0f9dffa2dc8e7292fc45360d61.

Simon

#6 lore raw
Quoted 10 lines · Simon Gaiser

Jason Andryuk:

Quoted 9 lines

A toolstack may delete the vif frontend and backend xenstore entries while xen-netfront is in the removal code path. In that case, the checks for xenbus_read_driver_state would return XenbusStateUnknown, and xennet_remove would hang indefinitely. This hang prevents system shutdown.

xennet_remove must be able to handle XenbusStateUnknown, and netback_changed must also wake up the wake_queue for that state as well.

Quoted 7 lines
Quoted 1 line
Fixes 5b5971df3bc2 ("xen-netfront: remove warning when unloading module")

I think this should go into stable since AFAIK the hanging network device can only be fixed by rebooting the guest. AFAICS this affects all 4.* branches since 5b5971df3bc2 got backported to them.

Upstream commit c2d2e6738a209f0f9dffa2dc8e7292fc45360d61.

Simon,

Yes, I agree. I actually submitted the request to stable earlier today, so hopefully it gets added soon.

Have you experienced this hang?

Regards,
Jason

#7 lore raw

Jason Andryuk:

Quoted 18 lines
Quoted 16 lines · Simon Gaiser

Jason Andryuk:

Quoted 10 lines

A toolstack may delete the vif frontend and backend xenstore entries while xen-netfront is in the removal code path. In that case, the checks for xenbus_read_driver_state would return XenbusStateUnknown, and xennet_remove would hang indefinitely. This hang prevents system shutdown.

xennet_remove must be able to handle XenbusStateUnknown, and netback_changed must also wake up the wake_queue for that state as well.

Fixes 5b5971df3bc2 ("xen-netfront: remove warning when unloading module")

I think this should go into stable since AFAIK the hanging network device can only be fixed by rebooting the guest. AFAICS this affects all 4.* branches since 5b5971df3bc2 got backported to them.

Quoted 6 lines
Quoted 1 line

Upstream commit c2d2e6738a209f0f9dffa2dc8e7292fc45360d61.

Simon,

Yes, I agree. I actually submitted the request to stable earlier today, so hopefully it gets added soon.

Ok, great. (I checked the stable patch queue, but didn't check the mailing list archive).

Quoted 1 line

Have you experienced this hang?

Yes, it's affecting the kernel shipped by Qubes OS (see [1]).

Thanks, Simon.

[1]: https://github.com/QubesOS/qubes-issues/issues/3657

#8 lore raw
Quoted 26 lines · Simon Gaiser

Jason Andryuk:

Quoted 18 lines
Quoted 16 lines · Simon Gaiser

Jason Andryuk:

Quoted 10 lines

A toolstack may delete the vif frontend and backend xenstore entries while xen-netfront is in the removal code path. In that case, the checks for xenbus_read_driver_state would return XenbusStateUnknown, and xennet_remove would hang indefinitely. This hang prevents system shutdown.

xennet_remove must be able to handle XenbusStateUnknown, and netback_changed must also wake up the wake_queue for that state as well.

Fixes 5b5971df3bc2 ("xen-netfront: remove warning when unloading module")

I think this should go into stable since AFAIK the hanging network device can only be fixed by rebooting the guest. AFAICS this affects all 4.* branches since 5b5971df3bc2 got backported to them.

Quoted 6 lines
Quoted 1 line

Upstream commit c2d2e6738a209f0f9dffa2dc8e7292fc45360d61.

Simon,

Yes, I agree. I actually submitted the request to stable earlier today, so hopefully it gets added soon.

Quoted 6 lines

Ok, great. (I checked the stable patch queue, but didn't check the mailing list archive).

Quoted 1 line

Have you experienced this hang?

Yes, it's affecting the kernel shipped by Qubes OS (see [1]).

Ok, interesting. I tracked down this bug with older xenvm tools, and I didn't know if libxl tools were also affected.

Greg KH added the patch to the stable queue, so it's in the process.

Regards,
Jason

#9 lore raw
In reply to Simon Gaiser
Quoted 26 lines · Simon Gaiser

Jason Andryuk:

Quoted 18 lines
Quoted 16 lines · Simon Gaiser

Jason Andryuk:

Quoted 10 lines

A toolstack may delete the vif frontend and backend xenstore entries while xen-netfront is in the removal code path. In that case, the checks for xenbus_read_driver_state would return XenbusStateUnknown, and xennet_remove would hang indefinitely. This hang prevents system shutdown.

xennet_remove must be able to handle XenbusStateUnknown, and netback_changed must also wake up the wake_queue for that state as well.

Fixes 5b5971df3bc2 ("xen-netfront: remove warning when unloading module")

I think this should go into stable since AFAIK the hanging network device can only be fixed by rebooting the guest. AFAICS this affects all 4.* branches since 5b5971df3bc2 got backported to them.

Quoted 6 lines
Quoted 1 line

Upstream commit c2d2e6738a209f0f9dffa2dc8e7292fc45360d61.

Simon,

Yes, I agree. I actually submitted the request to stable earlier today, so hopefully it gets added soon.

Quoted 6 lines

Ok, great. (I checked the stable patch queue, but didn't check the mailing list archive).

Quoted 1 line

Have you experienced this hang?

Yes, it's affecting the kernel shipped by Qubes OS (see [1]).

Ok, interesting. I tracked down this bug with older xenvm tools, and I didn't know if libxl tools were also affected.

Greg KH added the patch to the stable queue, so it's in the process.

Regards,
Jason

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#10 lore raw
In reply to Jason Andryuk

Jason Andryuk:

Quoted 18 lines
Quoted 16 lines · Simon Gaiser

Jason Andryuk:

Quoted 10 lines

A toolstack may delete the vif frontend and backend xenstore entries while xen-netfront is in the removal code path. In that case, the checks for xenbus_read_driver_state would return XenbusStateUnknown, and xennet_remove would hang indefinitely. This hang prevents system shutdown.

xennet_remove must be able to handle XenbusStateUnknown, and netback_changed must also wake up the wake_queue for that state as well.

Fixes 5b5971df3bc2 ("xen-netfront: remove warning when unloading module")

I think this should go into stable since AFAIK the hanging network device can only be fixed by rebooting the guest. AFAICS this affects all 4.* branches since 5b5971df3bc2 got backported to them.

Quoted 6 lines
Quoted 1 line

Upstream commit c2d2e6738a209f0f9dffa2dc8e7292fc45360d61.

Simon,

Yes, I agree. I actually submitted the request to stable earlier today, so hopefully it gets added soon.

Ok, great. (I checked the stable patch queue, but didn't check the mailing list archive).

Quoted 1 line

Have you experienced this hang?

Yes, it's affecting the kernel shipped by Qubes OS (see [1]).

Thanks, Simon.

[1]: https://github.com/QubesOS/qubes-issues/issues/3657

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#11 lore raw
In reply to Simon Gaiser
Quoted 10 lines · Simon Gaiser

Jason Andryuk:

Quoted 9 lines

A toolstack may delete the vif frontend and backend xenstore entries while xen-netfront is in the removal code path. In that case, the checks for xenbus_read_driver_state would return XenbusStateUnknown, and xennet_remove would hang indefinitely. This hang prevents system shutdown.

xennet_remove must be able to handle XenbusStateUnknown, and netback_changed must also wake up the wake_queue for that state as well.

Quoted 7 lines
Quoted 1 line
Fixes 5b5971df3bc2 ("xen-netfront: remove warning when unloading module")

I think this should go into stable since AFAIK the hanging network device can only be fixed by rebooting the guest. AFAICS this affects all 4.* branches since 5b5971df3bc2 got backported to them.

Upstream commit c2d2e6738a209f0f9dffa2dc8e7292fc45360d61.

Simon,

Yes, I agree. I actually submitted the request to stable earlier today, so hopefully it gets added soon.

Have you experienced this hang?

Regards,
Jason

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel

#12 lore raw
In reply to Jason Andryuk

Jason Andryuk:

Quoted 10 lines

A toolstack may delete the vif frontend and backend xenstore entries while xen-netfront is in the removal code path. In that case, the checks for xenbus_read_driver_state would return XenbusStateUnknown, and xennet_remove would hang indefinitely. This hang prevents system shutdown.

xennet_remove must be able to handle XenbusStateUnknown, and netback_changed must also wake up the wake_queue for that state as well.

Fixes 5b5971df3bc2 ("xen-netfront: remove warning when unloading module")

I think this should go into stable since AFAIK the hanging network device can only be fixed by rebooting the guest. AFAICS this affects all 4.* branches since 5b5971df3bc2 got backported to them.

Upstream commit c2d2e6738a209f0f9dffa2dc8e7292fc45360d61.

Simon

_______________________________________________
Xen-devel mailing list
[email protected]
https://lists.xenproject.org/mailman/listinfo/xen-devel