server/process: Handle SIGTERM more gracefully
PATCH 1/1 server/process: Handle SIGTERM more gracefully
Currently if you send a SIGTERM to the bitbake UI process, the system basically hangs if tasks are executing. This is because the server process doesn't actually try any kind of shutdown before exiting.
This patch trys executing a stateForceShutdown command first, which is enough to stop any active tasks before the system exits.
I also noticed that terminate can execute multiple times, once at SIGTERM from the handler and once from the real exit. Double execution leads to stack traces and potential hangs (writes to dead pipes), so ensure the code only can run once.
With these fixes, bitbake much more correctly deals with SIGTERM to the UI process.
bitbake/lib/bb/server/process.py +9 -0
@@ -114,6 +114,10 @@ class ProcessServer(Process, BaseImplServer): if self.quitout.poll(): self.quitout.recv() self.quit = True + try: + self.runCommand(["stateForceShutdown"]) + except: + pass self.idle_commands(.1, [self.command_channel, self.quitout]) except Exception:
@@ -123,6 +127,7 @@ class ProcessServer(Process, BaseImplServer): bb.event.unregister_UIHhandler(self.event_handle.value) self.command_channel.close() self.cooker.shutdown(True) + self.quitout.close() def idle_commands(self, delay, fds=None): nextsleep = delay
@@ -172,12 +177,16 @@ class BitBakeProcessServerConnection(BitBakeBaseServerConnection): self.event_queue = event_queue self.connection = ServerCommunicator(self.ui_channel, self.procserver.event_handle, self.procserver) self.events = self.event_queue + self.terminated = False def sigterm_terminate(self): bb.error("UI received SIGTERM") self.terminate() def terminate(self): + if self.terminated: + return + self.terminated = True def flushevents(): while True: try:
Quoted 13 lines · Richard Purdie
Currently if you send a SIGTERM to the bitbake UI process, the system basically
hangs if tasks are executing. This is because the server process doesn't actually try any kind of shutdown before exiting.This patch trys executing a stateForceShutdown command first, which is enough to stop any active tasks before the system exits.
I also noticed that terminate can execute multiple times, once at SIGTERM from the handler and once from the real exit. Double execution leads to stack traces and potential hangs (writes to dead pipes), so ensure the code only can run once.
Quoted 5 lines
With these fixes, bitbake much more correctly deals with SIGTERM to the UI process.
Signed-off-by Richard Purdie <[email protected]>
Just as an FYI for consideration in the longer term, you can raise an exception from a signal handler, to handle sigterm the same way as sigint for consistency. Ex.:
https://github.com/kergoth/oe-test-scripts/blob/master/bb_test.py#L12-L17 +
https://github.com/kergoth/oe-test-scripts/blob/master/bb_test.py#L171-L180
Which reminds me, we don't exit with the correct exit code for sigint/sigterm, so the calling shell doesn't know the cause of the exit. See also http://www.cons.org/cracauer/sigint.html -- fantastic article there, highly recommended.
https://docs.python.org/3/library/signal.html#example has an example of raising an exception from another signal handler, so it seems to be kosher.
Signature · 5 lines
Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics