bitbake: cooker: Kill alive process before join it
cover bitbake: cooker: Kill alive process before join it
The following changes since commit 58ab85d3e005476ab5bcc288ad421c0c5d1e0582:
systemd: add PACKAGECONFIG for gnu-efi (2019-08-21 21:52:59 +0100)
are available in the git repository at:
git://git.pokylinux.org/poky-contrib rbt/cooker
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rbt/cooker
Robert Yang (1):
bitbake: cooker: Kill alive process before join it
bitbake/lib/bb/cooker.py | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-)
Signature · 1 line
2.7.4
PATCH 1/1 bitbake: cooker: Kill alive process before join it
Fixed:
$ echo helloworld >> meta/recipes-extended/bash/bash_4.4.18.bb $ while true; do kill-bb; rm -fr bitbake-cookerdaemon.log tmp/cache/default-glibc/qemux86-64/x86_64/bb_cache.dat* ; bitbake -p; done
It may hang in 10 mins, there are two problems:
* There might be deadlocks when call process.join() if the queue is not NULL, so we need cleanup the queue before join() it, but:
* The self.result_queue.get(timeout=0.25) may hang if the queue._wlock is hold by SomeOtherProcess, the queue has the following info when it hangs:
'_wlock': <Lock(owner=SomeOtherProcess)>, so that we may can't clean up the queue.
We can kill alvie process before join it to fix the problems.
--- bitbake/lib/bb/cooker.py | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-)
bitbake/lib/bb/cooker.py +4 -8
@@ -2082,19 +2082,15 @@ class CookerParser(object): for process in self.processes: self.parser_quit.put(None) - # Cleanup the queue before call process.join(), otherwise there might be - # deadlocks. - while True: - try: - self.result_queue.get(timeout=0.25) - except queue.Empty: - break - for process in self.processes: if force: process.join(.1) process.terminate() else: + # Kill the alive process firstly before join() it to avoid + # deadlocks + if process.is_alive(): + os.kill(process.pid, 9) process.join() sync = threading.Thread(target=self.bb_cache.sync)
Signature · 1 line
2.7.4
Looks like you’re partially duplicating what the ‘force’ logic does already, but for the non-force case as well, yes? process.terminate() sounds a lot like sending a SIGTERM, though admittedly I haven’t read the code to Process recently. Perhaps if this is truly necessary, the logic can be simplified.
Quoted 55 lines · On Aug 22, 2019, 12:10 AM -0700, Robert Yang <[email protected]>, wrote:
Fixed:
$ echo helloworld >> meta/recipes-extended/bash/bash_4.4.18.bb $ while true; do kill-bb; rm -fr bitbake-cookerdaemon.log tmp/cache/default-glibc/qemux86-64/x86_64/bb_cache.dat* ; bitbake -p; doneIt may hang in 10 mins, there are two problems:
* There might be deadlocks when callprocess.join()if the queue is not NULL, so we need cleanup the queue beforejoin()it, but:
* Theself.result_queue.get(timeout=0.25)may hang if the queue._wlock is hold by SomeOtherProcess, the queue has the following info when it hangs:
'_wlock': <Lock(owner=SomeOtherProcess)>, so that we may can't clean up the queue.We can kill alvie process before join it to fix the problems.
Signed-off-by Robert Yang <[email protected]>---
bitbake/lib/bb/cooker.py| 12 ++++--------1 file changed, 4 insertions(+), 8 deletions(-)bitbake/lib/bb/cooker.py +0
-0@@ -2082,19 +2082,15 @@ class CookerParser(object):for process in self.processes:
self.parser_quit.put(None)- # Cleanup the queue before call
process.join(), otherwise there might be
- # deadlocks.
- while True:
- try:
-self.result_queue.get(timeout=0.25)
- except queue.Empty:
- break
-
for process in self.processes:
if force:process.join(.1)process.terminate()
else:
+ # Kill the alive process firstly beforejoin()it to avoid
+ # deadlocks
+ ifprocess.is_alive():
+os.kill(process.pid, 9)process.join()sync =
threading.Thread(target=self.bb_cache.sync)--
2.7.4--
_______________________________________________
bitbake-devel mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/bitbake-devel
Hi Larson,
Quoted 1 line · Christopher Larson
Looks like you’re partially duplicating what the ‘force’ logic does already, but for the non-force case as well, yes?
process.terminate()sounds a lot like sending a SIGTERM, though admittedly I haven’t read the code to Process recently. Perhaps if this is truly necessary, the logic can be simplified.
The process.join(.1) may also hang if the result_queue is not NULL, but AFAIK , this only happens when KeyboardInterrupt or Failed to parse a recipe, so I think that 'force' shutdown's process.jion(.1) won't hang, and I didn't see it hangs in the "force" code, we may have another way to make it better, check result_queue._wlock, but the _wlock is an internal variable, so I gave it up and just use os.kill() to avoid the hang problem completely, but there might be side effects that I didn't know?
// Robert
Quoted 56 lines · On Aug 22, 2019, 12:10 AM -0700, Robert Yang <[email protected]>, wrote:
Quoted 55 lines
Fixed:
$ echo helloworld >> meta/recipes-extended/bash/bash_4.4.18.bb $ while true; do kill-bb; rm -fr bitbake-cookerdaemon.log tmp/cache/default-glibc/qemux86-64/x86_64/bb_cache.dat* ; bitbake -p; doneIt may hang in 10 mins, there are two problems:
* There might be deadlocks when callprocess.join()if the queue is not NULL, so we need cleanup the queue beforejoin()it, but:
* Theself.result_queue.get(timeout=0.25)may hang if the queue._wlock is hold by SomeOtherProcess, the queue has the following info when it hangs:
'_wlock': <Lock(owner=SomeOtherProcess)>, so that we may can't clean up the queue.We can kill alvie process before join it to fix the problems.
Signed-off-by Robert Yang <[email protected]>---
bitbake/lib/bb/cooker.py| 12 ++++--------1 file changed, 4 insertions(+), 8 deletions(-)bitbake/lib/bb/cooker.py +0
-0@@ -2082,19 +2082,15 @@ class CookerParser(object):for process in self.processes:
self.parser_quit.put(None)- # Cleanup the queue before call
process.join(), otherwise there might be
- # deadlocks.
- while True:
- try:
-self.result_queue.get(timeout=0.25)
- except queue.Empty:
- break
-
for process in self.processes:
if force:process.join(.1)process.terminate()
else:
+ # Kill the alive process firstly beforejoin()it to avoid
+ # deadlocks
+ ifprocess.is_alive():
+os.kill(process.pid, 9)process.join()sync =
threading.Thread(target=self.bb_cache.sync)--
2.7.4--
_______________________________________________
bitbake-devel mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/bitbake-devel