Skip to contentSkip to conversation

Fix xenconsole after aborted save

PATCH In review Jiri Denemark 6 participants 11 messages Apr 20, 2009 – Apr 22, 2009

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Fix xenconsole after aborted save

Hi.

When a domain is resumed after aborted save (e.g., because of insufficient space on a device where the domain is being saved), xm console cannot send/read any data. The reason is that the event channel used by xenconsole stays unbound.

This patch modifies xenconsoled to check current status of open event channels and rebind them if necessary.

tools/console/daemon/io.c +24 -12
@@ -523,6 +523,7 @@ static int domain_create_ring(struct dom
 {
 	int err, remote_port, ring_ref, rc;
 	char *type, path[PATH_MAX];
+	int rebind = 0;
 
 	err = xs_gather(xs, dom->serialpath,
 			"ring-ref", "%u", &ring_ref,
@@ -548,8 +549,17 @@ static int domain_create_ring(struct dom
 	}
 	free(type);
 
-	if ((ring_ref == dom->ring_ref) && (remote_port == dom->remote_port))
-		goto out;
+	if ((ring_ref == dom->ring_ref) && (remote_port == dom->remote_port)) {
+		xc_evtchn_status_t status = {
+			.dom = DOMID_SELF,
+			.port = dom->local_port };
+
+		if (xc_evtchn_status(xc, &status)
+		    || status.status != EVTCHNSTAT_interdomain)
+			rebind = 1;
+		else
+			goto out;
+	}
 
 	if (ring_ref != dom->ring_ref) {
 		if (dom->interface != NULL)
@@ -565,17 +575,19 @@ static int domain_create_ring(struct dom
 		dom->ring_ref = ring_ref;
 	}
 
-	dom->local_port = -1;
-	dom->remote_port = -1;
-	if (dom->xce_handle != -1)
-		xc_evtchn_close(dom->xce_handle);
+	if (!rebind) {
+		dom->local_port = -1;
+		dom->remote_port = -1;
+		if (dom->xce_handle != -1)
+			xc_evtchn_close(dom->xce_handle);
 
-	/* Opening evtchn independently for each console is a bit
-	 * wasteful, but that's how the code is structured... */
-	dom->xce_handle = xc_evtchn_open();
-	if (dom->xce_handle == -1) {
-		err = errno;
-		goto out;
+		/* Opening evtchn independently for each console is a bit
+		 * wasteful, but that's how the code is structured... */
+		dom->xce_handle = xc_evtchn_open();
+		if (dom->xce_handle == -1) {
+			err = errno;
+			goto out;
+		}
 	}
  
 	rc = xc_evtchn_bind_interdomain(dom->xce_handle,

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel

#1 lore raw
Quoted 9 lines · Jiri Denemark

Hi.

When a domain is resumed after aborted save (e.g., because of insufficient space on a device where the domain is being saved), xm console cannot send/read any data. The reason is that the event channel used by xenconsole stays unbound.

This patch modifies xenconsoled to check current status of open event channels and rebind them if necessary.

close() + open() is the sledge hammer approach (will work though). Just unbind(local_port) should be enough.

cheers,
Gerd

#2 lore raw
Quoted 8 lines · Gerd Hoffmann
Quoted 7 lines · Jiri Denemark

Hi.

When a domain is resumed after aborted save (e.g., because of insufficient space on a device where the domain is being saved), xm console cannot send/read any data. The reason is that the event channel used by xenconsole stays unbound.

Quoted 5 lines
Quoted 2 lines

This patch modifies xenconsoled to check current status of open event channels and rebind them if necessary.

close() + open() is the sledge hammer approach (will work though). Just unbind(local_port) should be enough.

It doesn't close() and open(), it just calls xc_evtchn_bind_interdomain() in case the event channel is unbound. The close() + open() combination was there before... I haven't touched that code except for skipping it when only rebind is required.

#3 lore raw
Quoted 11 lines · Jiri Denemark
Quoted 5 lines · Gerd Hoffmann
Quoted 2 lines · Jiri Denemark

This patch modifies xenconsoled to check current status of open event channels and rebind them if necessary.

close() + open() is the sledge hammer approach (will work though). Just unbind(local_port) should be enough.

It doesn't close() and open(), it just calls xc_evtchn_bind_interdomain() in case the event channel is unbound. The close() + open() combination was there before... I haven't touched that code except for skipping it when only rebind is required.

Oh, ok, got the logic wrong. The patch looks fine to me then.

cheers,
Gerd

#4 lore raw
In reply to Jiri Denemark
Quoted 11 lines · Jiri Denemark
Quoted 6 lines
Quoted 3 lines

This patch modifies xenconsoled to check current status of open event channels
and rebind them if necessary.

close() + open() is the sledge hammer approach (will work though). Just unbind(local_port) should be enough.

It doesn't close() and open(), it just calls xc_evtchn_bind_interdomain() in case the event channel is unbound. The close() + open() combination was there before... I haven't touched that code except for skipping it when only rebind is required.

And actually that is a bug, since you will leak the old dom->local_port. I checked in an alternative patch as c/s 19561, so please take a look and test that resolves your issue.

Another thing to note is I think this problem can only occur if the domU does not support suspend cancellation (advertised as SUSPEND_CANCEL in kernel elf notes -- see xen/xend/XendDomainInfo.py:resumeDomain()). Your kernels should support that feature -- suspend cancellation (a.k.a. Resume) is very likely to be hit-or-miss without it!

-- Keir

#5 lore raw
Quoted 6 lines
Quoted 4 lines

It doesn't close() and open(), it just calls xc_evtchn_bind_interdomain() in case the event channel is unbound. The close() + open() combination was there before... I haven't touched that code except for skipping it when only rebind is required.

And actually that is a bug, since you will leak the old dom->local_port.

Ah, ok, thanks for fixing that...

Quoted 2 lines

I checked in an alternative patch as c/s 19561, so please take a look and test that resolves your issue.

Yes, the alternative patch works fine.

Quoted 2 lines

Another thing to note is I think this problem can only occur if the domU does not support suspend cancellation

Sure, the problem only occurs for kernels without SUSPEND_CANCEL feature.

Jirka

#6 lore raw
In reply to Keir Fraser
Quoted 5 lines · Keir Fraser

Another thing to note is I think this problem can only occur if the domU does not support suspend cancellation (advertised as SUSPEND_CANCEL in kernel elf notes -- see xen/xend/XendDomainInfo.py:resumeDomain()). Your kernels should support that feature -- suspend cancellation (a.k.a. Resume) is very likely to be hit-or-miss without it!

Could you elaborate a bit on this? I was under the impression that suspend cancellation was there mostly for the netaccel bits, but I have to admit I didn't look at it very closely. What scenarios do the suspend cancel bits help?

Signature · 1 line
Chris Lalancette
#7 lore raw
Quoted 11 lines · Chris Lalancette
Quoted 5 lines · Keir Fraser

Another thing to note is I think this problem can only occur if the domU does not support suspend cancellation (advertised as SUSPEND_CANCEL in kernel elf notes -- see xen/xend/XendDomainInfo.py:resumeDomain()). Your kernels should support that feature -- suspend cancellation (a.k.a. Resume) is very likely to be hit-or-miss without it!

Could you elaborate a bit on this? I was under the impression that suspend cancellation was there mostly for the netaccel bits, but I have to admit I didn't look at it very closely. What scenarios do the suspend cancel bits help?

Suspend failures (failure to save or to migrate). Also for live checkpointing/snapshotting. The feature indicates that the guest is happy for the suspend hypercall to return indicating 'failure/cancelled' and in which case it can pretty much resume whatever it was doing without any of the usual resume logic. The alternative is for the toolstack to make it look like the domain has been restored/migrated, by resetting PV devices and the like, and this is not much tested and almost inherently fragile.

The suspend_cancel hook you are thinking of is for any PV devices which *do* need to know that a suspend was cancelled. Netaccel does for some reason which I cannot recall.

Anyway, it is pretty easy and pretty important to support SUSPEND_CANCEL!

-- Keir

#8 lore raw
Quoted 1 line · Keir Fraser

Anyway, it is pretty easy and pretty important to support SUSPEND_CANCEL!

Note that it's still the case that Solaris doesn't, unfortunately.

regards
john

#9 lore raw
In reply to Keir Fraser
Quoted 21 lines · Keir Fraser
Quoted 10 lines · Chris Lalancette
Quoted 5 lines · Keir Fraser

Another thing to note is I think this problem can only occur if the domU does not support suspend cancellation (advertised as SUSPEND_CANCEL in kernel elf notes -- see xen/xend/XendDomainInfo.py:resumeDomain()). Your kernels should support that feature -- suspend cancellation (a.k.a. Resume) is very likely to be hit-or-miss without it!

Could you elaborate a bit on this? I was under the impression that suspend cancellation was there mostly for the netaccel bits, but I have to admit I didn't look at it very closely. What scenarios do the suspend cancel bits help?

Suspend failures (failure to save or to migrate). Also for live checkpointing/snapshotting. The feature indicates that the guest is happy for the suspend hypercall to return indicating 'failure/cancelled' and in which case it can pretty much resume whatever it was doing without any of the usual resume logic. The alternative is for the toolstack to make it look like the domain has been restored/migrated, by resetting PV devices and the like, and this is not much tested and almost inherently fragile.

Quoted 5 lines

The suspend_cancel hook you are thinking of is for any PV devices which *do* need to know that a suspend was cancelled. Netaccel does for some reason which I cannot recall.

Anyway, it is pretty easy and pretty important to support SUSPEND_CANCEL!

OK, that makes sense. Thanks Keir!

Signature · 1 line
Chris Lalancette
#10 lore raw
In reply to Keir Fraser
Quoted 3 lines · Keir Fraser

The suspend_cancel hook you are thinking of is for any PV devices which *do* need to know that a suspend was cancelled. Netaccel does for some reason which I cannot recall.

For the record, netaccel needs to remove any mappings of the hardware during the suspend callback since those mappings are not valid after the suspend has completed. Those hardware mappings (or the mappings appropriate for the new machine) are restored on resume as a result of the standard netfront/netback connection being re-established. If the suspend is cancelled, the netfront/netback connection stays in place but the hardware mappings need to be re-established which is why the suspend_cancel callback is used.

Cheers, Neil.