command/runqueue: Fix shutdown logic

If you hit Ctrl+C at the right point, the system processes the request but merrily continues building. It turns out finish_runqueue() is called but this doesn't stop the later generation and execution of the runqueue.

This patch adjusts some of the conditionals to ensure the build really does stop.

Signed-off-by Richard Purdie <[email protected]>

bitbake/lib/bb/command.py +1 -1

index 84fcdf9..d797fcf 100644
@@ -86,7 +86,7 @@ class Command:
 
     def runAsyncCommand(self):
         try:
-            if self.cooker.state == bb.cooker.state.error:
+            if self.cooker.state in (bb.cooker.state.error, bb.cooker.state.shutdown, bb.cooker.state.forceshutdown):
                 return False
             if self.currentAsyncCommand is not None:
                 (command, options) = self.currentAsyncCommand

bitbake/lib/bb/runqueue.py +2 -1

index 4ea4970..f68a11d 100644
@@ -1065,7 +1065,7 @@ class RunQueue:
         if self.state is runQueueCleanUp:
            self.rqexe.finish()
 
-        if self.state is runQueueComplete or self.state is runQueueFailed:
+        if (self.state is runQueueComplete or self.state is runQueueFailed) and self.rqexe:
             self.teardown_workers()
             if self.rqexe.stats.failed:
                 logger.info("Tasks Summary: Attempted %d tasks of which %d didn't need to be rerun and %d failed.", self.rqexe.stats.completed + self.rqexe.stats.failed, self.rqexe.stats.skipped, self.rqexe.stats.failed)

@@ -1106,6 +1106,7 @@ class RunQueue:
 
     def finish_runqueue(self, now = False):
         if not self.rqexe:
+            self.state = runQueueComplete
             return
 
         if now:

Signature · 9 lines

cgit v0.10.1



-- 
_______________________________________________
bitbake-devel mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/bitbake-devel

Quoted 7 lines · Richard Purdie

If you hit Ctrl+C at the right point, the system processes the request but merrily continues building. It turns out finish_runqueue() is called but this doesn't stop the later generation and execution of the runqueue.

This patch adjusts some of the conditionals to ensure the build really does stop.

Great, I've included this change in my world builds to see if it fixes bitbake still running after jenkins job is aborted.

I don't think it's caused by this change and I don't know how much we can do about it, but today I was testing snort build (which eats all memory in m4 call until OOMK kills it)

and when I wanted to interrupt
the build it failed with 2 tracebacks:

NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Running task 569 of 610 (ID: 5, /OE/build/oe-core/meta-openembedded/meta-networking/recipes-connectivity/snort/snort_2.9.6.0.bb, do_configure)
NOTE: recipe snort-2.9.6.0-r0: task do_configure: Started
^C^C^C^C^C^C^C^C^C^C^C^C^C^CTraceback (most recent call last):
File "/OE/build/oe-core/bitbake/bin/bitbake", line 382, in <module>
ret = main()
File "/OE/build/oe-core/bitbake/bin/bitbake", line 372, in main

    bb.event.ui_queue = []

KeyboardInterrupt
^CException KeyboardInterrupt in <module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored

^CError in atexit._run_exitfuncs:
^CError in sys.exitfunc:
Traceback (most recent call last):
File "/usr/lib64/python2.7/atexit.py", line 30, in _run_exitfuncs
traceback.print_exc()
File "/usr/lib64/python2.7/traceback.py", line 233, in print_exc
print_exception(etype, value, tb, limit, file)
File "/usr/lib64/python2.7/traceback.py", line 110, in print_exception
def print_exception(etype, value, tb, limit=None, file=None):
KeyboardInterrupt

There was also about 5 minute delay between first 2 Ctrl+C and actual exit, but that could be caused by huge load caused by that faulty m4.

Quoted 11 lines · Martin Jansa

Quoted 7 lines · Richard Purdie

If you hit Ctrl+C at the right point, the system processes the request but merrily continues building. It turns out finish_runqueue() is called but this doesn't stop the later generation and execution of the runqueue.

This patch adjusts some of the conditionals to ensure the build really does stop.

Great, I've included this change in my world builds to see if it fixes bitbake still running after jenkins job is aborted.

I've seen that too and I don't think this fix will address that unfortunately. Its on my list of things to look into.

Quoted 3 lines

I don't think it's caused by this change and I don't know how much we can do about it, but today I was testing snort build (which eats all memory in m4 call until OOMK kills it)

I ended up excluding snort from my builds for that reason. Builds go a lot faster when its not destroying the machine!

Quoted 27 lines

and when I wanted to interrupt
the build it failed with 2 tracebacks:

NOTE: Preparing runqueue
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Running task 569 of 610 (ID: 5, /OE/build/oe-core/meta-openembedded/meta-networking/recipes-connectivity/snort/snort_2.9.6.0.bb, do_configure)
NOTE: recipe snort-2.9.6.0-r0: task do_configure: Started
^C^C^C^C^C^C^C^C^C^C^C^C^C^CTraceback (most recent call last):
File "/OE/build/oe-core/bitbake/bin/bitbake", line 382, in <module>
ret = main()
File "/OE/build/oe-core/bitbake/bin/bitbake", line 372, in main

    bb.event.ui_queue = []

KeyboardInterrupt
^CException KeyboardInterrupt in <module 'threading' from '/usr/lib64/python2.7/threading.pyc'> ignored

^CError in atexit._run_exitfuncs:
^CError in sys.exitfunc:
Traceback (most recent call last):
File "/usr/lib64/python2.7/atexit.py", line 30, in _run_exitfuncs
traceback.print_exc()
File "/usr/lib64/python2.7/traceback.py", line 233, in print_exc
print_exception(etype, value, tb, limit, file)
File "/usr/lib64/python2.7/traceback.py", line 110, in print_exception
def print_exception(etype, value, tb, limit=None, file=None):
KeyboardInterrupt

Quoted 2 lines

There was also about 5 minute delay between first 2 Ctrl+C and actual exit, but that could be caused by huge load caused by that faulty m4.

Thanks, I'll have a look at those and see if they're significant and if we can do anything about them. It may be they are "one offs" and unlikely to reproduce but we'll see.

Cheers,

Richard