xend: Sleep before sending SIGKILL to device model
PATCH 1/1 xend: Sleep before sending SIGKILL to device model
When we destroy a domain, xend sends SIGTERM to the device model and wait by waitpid() until the device model process disappears. If we restarted xend during the lifetime of the domain, waitpid() fails because the device model is no longer a child of xend, and in that case xend gives up waiting for the shutdown of process and just send it SIGKILL immediately. This is problematic because most of the case the device model will be forcibly killed by xend before shutting itself down.
This patch adds time.sleep before sending SIGKILL to the device model. On my test box shutdown of a device model usually takes about 0.5 sec, so waiting two seconds should be enough in most cases.
Regards,
-----------------------
Yosuke Iwamatsu
NEC Corporation
xend: Sleep before sending SIGKILL to device model
tools/python/xen/xend/image.py +3 -1
@@ -574,7 +574,9 @@ # This is expected if Xend has been restarted within the # life of this domain. In this case, we can kill the process, # but we can't wait for it because it's not our child. - # We just make really sure it's going away (SIGKILL) first. + # We just sleep for a reasonable period of time and + # make really sure it's going away (SIGKILL) first. + time.sleep(2) os.kill(self.pid, signal.SIGKILL) state = xstransact.Remove("/local/domain/0/device-model/%i" % self.vm.getDomid())
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Yosuke Iwamatsu writes ("[Xen-devel] [PATCH] xend: Sleep before sending SIGKILL to device model"):
Quoted 8 lines
When we destroy a domain, xend sends SIGTERM to the device model and wait by
waitpid()until the device model process disappears. If we restarted xend during the lifetime of the domain,waitpid()fails because the device model is no longer a child of xend, and in that case xend gives up waiting for the shutdown of process and just send it SIGKILL immediately. This is problematic because most of the case the device model will be forcibly killed by xend before shutting itself down.
The code already has a timeout to forcibly kill the device model after (I think) 10 seconds. Surely we should reuse that code path (and the same timeout value) ?
Restarting xend is not a usual thing to do and I think it's OK if shutting down a domain started by a previous xend involves waiting for such a longer timeout. It's better to err on the side of safety.
Also, your patch was:
Content-Type: all/allfiles;
This is not a recognised content type and prevented both of my mailreaders from displaying it to me. Can you please fix your MUA ?
Alternatively, just include the patch in the body of the mail rather than attaching it.
Ian.
PATCH 1/1 xend: Sleep before sending SIGKILL to device model
Quoted 7 lines · Ian Jackson
The code already has a timeout to forcibly kill the device model after (I think) 10 seconds. Surely we should reuse that code path (and the same timeout value) ?
Restarting xend is not a usual thing to do and I think it's OK if shutting down a domain started by a previous xend involves waiting for such a longer timeout. It's better to err on the side of safety.
O.K. Attached is a revised patch which reuses the existing code path. 10 seconds seems to me a bit too long, but I can agree we had better keep on the safe side.
Quoted 4 lines
Also, your patch was:
Content-Type: all/allfiles;
This is not a recognised content type and prevented both of my mailreaders from displaying it to me. Can you please fix your MUA ?
Sorry for the inconvenience.
This time your mail client can recognize it, I think.
-- Yosuke
tools/python/xen/xend/image.py +19 -13
@@ -558,24 +558,30 @@ os.kill(self.pid, signal.SIGHUP) except OSError, exn: log.exception(exn) - try: - # Try to reap the child every 100ms for 10s. Then SIGKILL it. - for i in xrange(100): + # Try to reap the child every 100ms for 10s. Then SIGKILL it. + for i in xrange(100): + try: (p, rv) = os.waitpid(self.pid, os.WNOHANG) if p == self.pid: break - time.sleep(0.1) - else: - log.warning("DeviceModel %d took more than 10s " - "to terminate: sending SIGKILL" % self.pid) + except OSError: + # This is expected if Xend has been restarted within + # the life of this domain. In this case, we can kill + # the process, but we can't wait for it because it's + # not our child. We continue this loop, and after it is + # terminated make really sure the process is going away + # (SIGKILL). + pass + time.sleep(0.1) + else: + log.warning("DeviceModel %d took more than 10s " + "to terminate: sending SIGKILL" % self.pid) + try: os.kill(self.pid, signal.SIGKILL) os.waitpid(self.pid, 0) - except OSError, exn: - # This is expected if Xend has been restarted within the - # life of this domain. In this case, we can kill the process, - # but we can't wait for it because it's not our child. - # We just make really sure it's going away (SIGKILL) first. - os.kill(self.pid, signal.SIGKILL) + except OSError: + # This happens if the process doesn't exist. + pass state = xstransact.Remove("/local/domain/0/device-model/%i" % self.vm.getDomid()) finally:
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel
Yosuke Iwamatsu writes ("[Xen-devel] [PATCH] xend: Sleep before sending SIGKILL to device model"):
Quoted 3 lines
O.K. Attached is a revised patch which reuses the existing code path. 10 seconds seems to me a bit too long, but I can agree we had better keep on the safe side.
That looks reasonable to me, thanks. (Although I haven't tested it.)
Quoted 1 line
Signed-off-by Yosuke Iwamatsu <[email protected]>
Ian.
PATCH 1/1 xend: Sleep before sending SIGKILL to device model
In reply to Yosuke IwamatsuKeir,
Would you mind applying this?
-- Yosuke
Quoted 34 lines · Yosuke Iwamatsu
Quoted 7 lines · Ian Jackson
The code already has a timeout to forcibly kill the device model after (I think) 10 seconds. Surely we should reuse that code path (and the same timeout value) ?
Restarting xend is not a usual thing to do and I think it's OK if shutting down a domain started by a previous xend involves waiting for such a longer timeout. It's better to err on the side of safety.
O.K. Attached is a revised patch which reuses the existing code path. 10 seconds seems to me a bit too long, but I can agree we had better keep on the safe side.
Quoted 4 lines
Also, your patch was:
Content-Type: all/allfiles;
This is not a recognised content type and prevented both of my mailreaders from displaying it to me. Can you please fix your MUA ?Sorry for the inconvenience.
This time your mail client can recognize it, I think.-- Yosuke
Signed-off-by Yosuke Iwamatsu <[email protected]>------------------------------------------------------------------------
_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xensource.com/xen-devel