daemonize: Flush stdio on exit

In spite of a comment suggesting otherwise, os._exit() does not flush buffered output from file descriptors before exiting the process like os.exit() does. This means that any un-flushed output is lost in the daemon process, in particular the traceback from any thrown exceptions, making debugging exceptions in the daemon quite difficult.

The solution is to flush stdout and stderr before exiting.

Signed-off-by Joshua Watt <[email protected]>

---
 bitbake/lib/bb/daemonize.py | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

bitbake/lib/bb/daemonize.py +2 -2

index 8300d1d0f0f..bf16793468e 100644
@@ -49,8 +49,8 @@ def createDaemon(function, logfile):
             # exit() or _exit()?
             # _exit is like exit(), but it doesn't call any functions registered
             # with atexit (and on_exit) or any registered signal handlers.  It also
-            # closes any open file descriptors.  Using exit() may cause all stdio
-            # streams to be flushed twice and any temporary files may be unexpectedly
+            # closes any open file descriptors, but doesn't flush any buffered output.
+            # Using exit() may cause all any temporary files to be unexpectedly
             # removed.  It's therefore recommended that child branches of a fork()
             # and the parent branch(es) of a daemon use _exit().
             os._exit(0)

Quoted 52 lines · Joshua Watt

In spite of a comment suggesting otherwise, os._exit() does not flush
buffered output from file descriptors before exiting the process like
os.exit() does. This means that any un-flushed output is lost in the
daemon process, in particular the traceback from any thrown exceptions,
making debugging exceptions in the daemon quite difficult.

The solution is to flush stdout and stderr before exiting.

Signed-off-by Joshua Watt <[email protected]>

---
 bitbake/lib/bb/daemonize.py | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

bitbake/lib/bb/daemonize.py +0 -0

index 8300d1d0f0f..bf16793468e 100644
@@ -49,8 +49,8 @@ def createDaemon(function, logfile):
             # exit() or _exit()?
             # _exit is like exit(), but it doesn't call any

functions registered

             # with atexit (and on_exit) or any registered signal
handlers.  It also
-            # closes any open file descriptors.  Using exit() may
cause all stdio
-            # streams to be flushed twice and any temporary files
may be unexpectedly
+            # closes any open file descriptors, but doesn't flush
any buffered output.
+            # Using exit() may cause all any temporary files to be
unexpectedly
             # removed.  It's therefore recommended that child
branches of a fork()
             # and the parent branch(es) of a daemon use _exit().
             os._exit(0)

bitbake/lib/bb/daemonize.py +1 -0

@@ -79,4 +79,9 @@ def createDaemon(function, logfile):
         traceback.print_exc()
     finally:
         bb.event.print_ui_queue()
+        # os._exit() doesn't flush open files like os.exit() does.

Manually flush
+ # stdout and stderr so that any logging output will be
seen, particularly

bitbake/lib/bb/daemonize.py +3 -0

+        # exception tracebacks.
+        sys.stdout.flush()
+        sys.stderr.flush()
         os._exit(0)

Quoted 1 line

Well, that's of course better than not getting the output, but there's then the issue of double printing: The intermediate child does _exit(), but the grandparent may have had stuff in the stdio buffers before the very first fork(), and that content would be inherited down to the grandchild. So I think you should (also) explicitly flush stdout and stderr before the first fork().

Quoted 6 lines · Rasmus Villemoes

Well, that's of course better than not getting the output, but there's then the issue of double printing: The intermediate child does _exit(), but the grandparent may have had stuff in the stdio buffers before the very first fork(), and that content would be inherited down to the grandchild. So I think you should (also) explicitly flush stdout and stderr before the first fork().

You're correct, we should do that. I was thinking we did flush things before forking but looking at the code I'm thinking of the fork() call
in bitbake-worker, not the one in daemonize.py.

I'd happily take a patch if someone wants to work one up!