Skip to contentSkip to conversation

bitbake-server does not exit after build ends

DISCUSSION Böszörményi Zoltán 7 participants 21 messages Nov 27, 2023 – Jan 8, 2024 2 messages not in the archive

lore
Share & tools
lore
#0 lore raw

Hi,

I am not sure what's the actual problem here.

FWIW, I have recently upgraded my laptop to Fedora 39 where I test recipe changes first.

Now, whenever a build ends (no matter if successfully or if there was a recipe build failure) two bitbake-server processes stay running.

As a result, the next bitbake run spits out messages about the previous instance holding bitbake.lock and retries.

Fedora 39 has Python 3.12 but maybe an outdated python module is the issue.

I am happy to try any ideas.

Thanks in advance,
Zoltán

#1 lore raw

Fedora 39 isn't yet supported or tested, so it's almost certainly you've hit a python 3.12 compatibility issue. If you can look into, that would be very appreciated.

Alex

Quoted 32 lines · Zoltan Boszormenyi

Hi,

I am not sure what's the actual problem here.

FWIW, I have recently upgraded my laptop to Fedora 39 where I test recipe changes first.

Now, whenever a build ends (no matter if successfully or if there was a recipe build failure) two bitbake-server processes stay running.

As a result, the next bitbake run spits out messages about the previous instance holding bitbake.lock and retries.

Fedora 39 has Python 3.12 but maybe an outdated python module is the issue.

I am happy to try any ideas.

Thanks in advance,
Zoltán

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#191263): https://lists.openembedded.org/g/openembedded-core/message/191263
Mute This Topic: https://lists.openembedded.org/mt/102826186/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#2 lore raw
In reply to Böszörményi Zoltán
Quoted 11 lines · Zoltan Boszormenyi

Hi,

I am not sure what's the actual problem here.

FWIW, I have recently upgraded my laptop to Fedora 39 where I test recipe changes first.

Now, whenever a build ends (no matter if successfully or if there was a recipe build failure) two bitbake-server processes stay running.

Quoted 7 lines

As a result, the next bitbake run spits out messages about the previous instance holding bitbake.lock and retries.

Fedora 39 has Python 3.12 but maybe an outdated python module is the issue.

I am happy to try any ideas.

Is there anything interesting in the bitbake-cookerdaemon.log file?

Cheers,

Richard

#3 lore raw

2023. 11. 27. 11:52 keltezéssel, Richard Purdie írta:

Quoted 1 line

Is there anything interesting in the bitbake-cookerdaemon.log file?

Actually, yes.

This is obviously the most suspicious:

/usr/lib64/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning: This process (pid=1605202) is multi-threaded, use of fork() may lead to deadlocks in the child. self.pid = os.fork()
/home/zozo/test-yocto-4.3/bitbake/bin/bitbake-worker:210: DeprecationWarning: This process (pid=1605590) is multi-threaded, use of fork() may lead to deadlocks in the child. pid = os.fork()
/home/zozo/test-yocto-4.3/bitbake/bin/bitbake-worker:210: DeprecationWarning: This process (pid=1607833) is multi-threaded, use of fork() may lead to deadlocks in the child. pid = os.fork()

There are a few other deprecation warnings, too, which are not relevant to this particular problem but may be interesting for the future, nevertheless:

/home/zozo/test-yocto-4.3/bitbake/lib/bb/codeparser.py:260: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
/home/zozo/test-yocto-4.3/bitbake/lib/bb/codeparser.py:263: DeprecationWarning: Attribute s is deprecated and will be removed in Python 3.14; use value instead self.contains[varname].add(node.args[1].s)

ruby.bbclass has a lot of these at different line numbers:

/home/zozo/test-yocto-4.3/conf/../layers/meta-cloud-services/classes/ruby.bbclass:28: SyntaxWarning: invalid escape sequence '\.' r = re.compile("ruby ([0-9]+\.[0-9]+\.[0-9]+)*")

These lines occur at the end (probably normal):

1605202 10:39:58.854287 Parse cache valid
1605202 10:40:13.098694 Registering idle function <function BBCooker.buildTargets.<locals>.buildTargetsIdle at 0x7fa3e22e79c0> 1605202 10:40:13.098736 Removing idle function <bound method Command.runAsyncCommand of <bb.command.Command object at 0x7fa41259d580>> 1605202 12:17:02.357378 Running command ['stateShutdown']
1605202 12:17:02.357622 Sending reply (None, None)
1605202 12:17:02.357819 Command Completed (socket: True)

And the remaining child processes:

$ ps auxw | grep bitb | grep -v grep
zozo     1605202  2.3  3.5 1333384 1151808 ?     Sl   10:39   2:29 /usr/bin/python3 /home/zozo/test-yocto-4.3/bitbake/bin/bitbake-server decafbad 3 5 /home/zozo/test-yocto-4.3/bitbake-cookerdaemon.log /home/zozo/test-yocto-4.3/bitbake.lock /home/zozo/test-yocto-4.3/bitbake.sock 0 0 None 0
zozo     1605208  0.0  1.4 559372 466260 ?       S    10:39   0:00 /usr/bin/python3 /home/zozo/test-yocto-4.3/bitbake/bin/bitbake-server decafbad 3 5 /home/zozo/test-yocto-4.3/bitbake-cookerdaemon.log /home/zozo/test-yocto-4.3/bitbake.lock /home/zozo/test-yocto-4.3/bitbake.sock 0 0 None 0
#4 lore raw

I can confirm the same on gentoo with python-3.12.

Quoted 83 lines · Zoltan Boszormenyi

2023. 11. 27. 11:52 keltezéssel, Richard Purdie írta:

Quoted 1 line

Is there anything interesting in the bitbake-cookerdaemon.log file?

Actually, yes.

This is obviously the most suspicious:

/usr/lib64/python3.12/multiprocessing/popen_fork.py:66:
DeprecationWarning: This process
(pid=1605202) is multi-threaded, use of fork() may lead to deadlocks in the child.
self.pid = os.fork()
/home/zozo/test-yocto-4.3/bitbake/bin/bitbake-worker:210:
DeprecationWarning: This process
(pid=1605590) is multi-threaded, use of fork() may lead to deadlocks in the child.
pid = os.fork()
/home/zozo/test-yocto-4.3/bitbake/bin/bitbake-worker:210:
DeprecationWarning: This process
(pid=1607833) is multi-threaded, use of fork() may lead to deadlocks in the child.
pid = os.fork()

There are a few other deprecation warnings, too, which are not relevant to this particular problem but may be interesting for the future, nevertheless:

/home/zozo/test-yocto-4.3/bitbake/lib/bb/codeparser.py:260:
DeprecationWarning: ast.Str is
deprecated and will be removed in Python 3.14; use ast.Constant instead
if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
/home/zozo/test-yocto-4.3/bitbake/lib/bb/codeparser.py:263:
DeprecationWarning: Attribute
s is deprecated and will be removed in Python 3.14; use value instead
self.contains[varname].add(node.args[1].s)

ruby.bbclass has a lot of these at different line numbers:

/home/zozo/test-yocto-4.3/conf/../layers/meta-cloud-services/classes/ruby.bbclass:28:

SyntaxWarning: invalid escape sequence '\.'
r = re.compile("ruby ([0-9]+\.[0-9]+\.[0-9]+)*")

These lines occur at the end (probably normal):

1605202 10:39:58.854287 Parse cache valid
1605202 10:40:13.098694 Registering idle function <function BBCooker.buildTargets.<locals>.buildTargetsIdle at 0x7fa3e22e79c0> 1605202 10:40:13.098736 Removing idle function <bound method Command.runAsyncCommand of
<bb.command.Command object at 0x7fa41259d580>>
1605202 12:17:02.357378 Running command ['stateShutdown'] 1605202 12:17:02.357622 Sending reply (None, None) 1605202 12:17:02.357819 Command Completed (socket: True)

And the remaining child processes:

$ ps auxw | grep bitb | grep -v grep
zozo     1605202  2.3  3.5 1333384 1151808 ?     Sl   10:39   2:29
/usr/bin/python3
/home/zozo/test-yocto-4.3/bitbake/bin/bitbake-server decafbad 3 5
/home/zozo/test-yocto-4.3/bitbake-cookerdaemon.log
/home/zozo/test-yocto-4.3/bitbake.lock
/home/zozo/test-yocto-4.3/bitbake.sock 0 0 None 0
zozo     1605208  0.0  1.4 559372 466260 ?       S    10:39   0:00
/usr/bin/python3
/home/zozo/test-yocto-4.3/bitbake/bin/bitbake-server decafbad 3 5
/home/zozo/test-yocto-4.3/bitbake-cookerdaemon.log
/home/zozo/test-yocto-4.3/bitbake.lock
/home/zozo/test-yocto-4.3/bitbake.sock 0 0 None 0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#191294):
https://lists.openembedded.org/g/openembedded-core/message/191294
Mute This Topic: https://lists.openembedded.org/mt/102826186/3617156
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#5 lore raw

So it's not just Fedora, but Python 3.12 in general.

Googling for problem with threads + fork reveals that it is indeed an issue even in C / POSIX: only the thread executing fork() will become part of the new process, but not the main process starting the thread or any other threads from the original process.

The warning is justified, and apparently the issue not occurring in bitbake with previous Python versions is pure luck.

2023. 11. 27. 14:14 keltezéssel, Martin Jansa írta:

Quoted 80 lines

I can confirm the same on gentoo with python-3.12.

On Mon, Nov 27, 2023 at 12:29 PM Zoltan Boszormenyi <[email protected]> wrote:

2023. 11. 27. 11:52 keltezéssel, Richard Purdie írta:
> Is there anything interesting in the bitbake-cookerdaemon.log file?

Actually, yes.

This is obviously the most suspicious:

/usr/lib64/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning: This process
(pid=1605202) is multi-threaded, use of fork() may lead to deadlocks in the child. self.pid = os.fork() /home/zozo/test-yocto-4.3/bitbake/bin/bitbake-worker:210: DeprecationWarning: This process
(pid=1605590) is multi-threaded, use of fork() may lead to deadlocks in the child. pid = os.fork() /home/zozo/test-yocto-4.3/bitbake/bin/bitbake-worker:210: DeprecationWarning: This process
(pid=1607833) is multi-threaded, use of fork() may lead to deadlocks in the child. pid = os.fork()

There are a few other deprecation warnings, too, which are not relevant to this particular problem but may be interesting for the future, nevertheless:

/home/zozo/test-yocto-4.3/bitbake/lib/bb/codeparser.py:260: DeprecationWarning:
ast.Str is
deprecated and will be removed in Python 3.14; use ast.Constant instead if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
/home/zozo/test-yocto-4.3/bitbake/lib/bb/codeparser.py:263: DeprecationWarning:
Attribute
s is deprecated and will be removed in Python 3.14; use value instead self.contains[varname].add(node.args[1].s)

ruby.bbclass has a lot of these at different line numbers:

/home/zozo/test-yocto-4.3/conf/../layers/meta-cloud-services/classes/ruby.bbclass:28:
SyntaxWarning: invalid escape sequence '\.'
   r = re.compile("ruby ([0-9]+\.[0-9]+\.[0-9]+)*")

These lines occur at the end (probably normal):

1605202 10:39:58.854287 Parse cache valid
1605202 10:40:13.098694 Registering idle function <function BBCooker.buildTargets.<locals>.buildTargetsIdle at 0x7fa3e22e79c0> 1605202 10:40:13.098736 Removing idle function <bound method Command.runAsyncCommand of <bb.command.Command object at 0x7fa41259d580>>
1605202 12:17:02.357378 Running command ['stateShutdown']
1605202 12:17:02.357622 Sending reply (None, None)
1605202 12:17:02.357819 Command Completed (socket: True)

And the remaining child processes:

$ ps auxw | grep bitb | grep -v grep
zozo     1605202  2.3  3.5 1333384 1151808 ?     Sl   10:39 2:29 /usr/bin/python3 /home/zozo/test-yocto-4.3/bitbake/bin/bitbake-server decafbad 3 5 /home/zozo/test-yocto-4.3/bitbake-cookerdaemon.log /home/zozo/test-yocto-4.3/bitbake.lock /home/zozo/test-yocto-4.3/bitbake.sock 0 0 None 0
zozo     1605208  0.0  1.4 559372 466260 ?       S    10:39 0:00 /usr/bin/python3 /home/zozo/test-yocto-4.3/bitbake/bin/bitbake-server decafbad 3 5 /home/zozo/test-yocto-4.3/bitbake-cookerdaemon.log /home/zozo/test-yocto-4.3/bitbake.lock /home/zozo/test-yocto-4.3/bitbake.sock 0 0 None 0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#191294):
https://lists.openembedded.org/g/openembedded-core/message/191294
Mute This Topic: https://lists.openembedded.org/mt/102826186/3617156
Group Owner: [email protected] <mailto:openembedded-core%[email protected]>
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#6 lore raw
Quoted 11 lines · Böszörményi Zoltán

So it's not just Fedora, but Python 3.12 in general.

Googling for problem with threads + fork reveals that it is indeed an issue even in C / POSIX: only the thread executing fork() will become part of the new process, but not the main process starting the thread or any other threads from the original process.

The warning is justified, and apparently the issue not occurring in bitbake with previous Python versions is pure luck.

I'm not sure that is entirely true/fair. The way bitbake has executed processes and handled threads was very carefully written deliberately to avoid some of these kinds of issues so it wasn't "pure luck". What I don't know is what changed in 3.12 to mean this now is potentially breaking.

Cheers,

Richard

#7 lore raw
Quoted 5 lines · Richard Purdie

I'm not sure that is entirely true/fair. The way bitbake has executed processes and handled threads was very carefully written deliberately to avoid some of these kinds of issues so it wasn't "pure luck". What I don't know is what changed in 3.12 to mean this now is potentially breaking.

With my python 3.12 recipe I'm getting a different issue when that recipe is used to produce python in buildtools, and much sooner:
https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/8593/steps/13/logs/stdio

I've reproduced it locally, and looking into it. It seems as though hash server is not creating the unix socket file, but isn't failing to start either.

Alex

#8 lore raw
In reply to Richard Purdie

2023. 11. 28. 11:00 keltezéssel, Richard Purdie írta:

Quoted 9 lines · Böszörményi Zoltán
Quoted 8 lines

So it's not just Fedora, but Python 3.12 in general.

Googling for problem with threads + fork reveals that it is indeed an issue even in C / POSIX: only the thread executing fork() will become part of the new process, but not the main process starting the thread or any other threads from the original process.

Quoted 4 lines
Quoted 3 lines

The warning is justified, and apparently the issue not occurring in bitbake with previous Python versions is pure luck.

I'm not sure that is entirely true/fair.

Maybe not fair. Still, true. See this piece:

http://www.doublersolutions.com/docs/dce/osfdocs/htmls/develop/appdev/Appde193.htm

Other discussions, like this one confirm the same:
https://stackoverflow.com/questions/39890363/what-happens-when-a-thread-forks

Quoted 5 lines

The way bitbake has executed
processes and handled threads was very carefully written deliberately to avoid some of these kinds of issues so it wasn't "pure luck". What I don't know is what changed in 3.12 to mean this now is potentially breaking.

The warning only occurs if os.fork() is called when there are already Python threads.

I guess the general performance uplift in Python 3.12 can trigger a pre-existing issue in bitbake more easily.

It is entirely possible that the threads bitbake creates do not exist yet with Python 3.11 and earlier at the time of executing os.fork(). ¯\_(ツ)_/¯

#9 lore raw
Quoted 9 lines · Zoltan Boszormenyi

The warning only occurs if os.fork() is called when there are already Python threads.

I guess the general performance uplift in Python 3.12 can trigger a pre-existing issue in bitbake more easily.

It is entirely possible that the threads bitbake creates do not exist yet with Python 3.11 and earlier at the time of executing os.fork(). ¯\_(ツ)_/¯

I think you might be too quick to jump to conclusions here. I've fixed the async issue I've been observing (backporting
https://github.com/python/cpython/commit/2e5d4e24ebcd671915427b6fd35a5b0eea412c4a
), and when running bitbake with 3.12 from buildtools-tarball, everything works fine, and I'm not seeing any leftover processes. Poky revision is current master:
https://git.yoctoproject.org/poky/commit/?h=master&id=3e50e45917831d9da2d86e69fb908a1a78483b62

Alex

#10 lore raw

2023. 11. 28. 15:50 keltezéssel, Alexander Kanavin írta:

Quoted 11 lines · Zoltan Boszormenyi
Quoted 9 lines

The warning only occurs if os.fork() is called when there are already Python threads.

I guess the general performance uplift in Python 3.12 can trigger a pre-existing issue in bitbake more easily.

It is entirely possible that the threads bitbake creates do not exist yet with Python 3.11 and earlier at the time of executing os.fork(). ¯\_(ツ)_/¯

I think you might be too quick to jump to conclusions here.

Indeed.
I should have added "assuming Python is not the buggy party here". Sorry.

Quoted 7 lines

I've fixed
the async issue I've been observing (backporting
https://github.com/python/cpython/commit/2e5d4e24ebcd671915427b6fd35a5b0eea412c4a
), and when running bitbake with 3.12 from buildtools-tarball, everything works fine, and I'm not seeing any leftover processes. Poky revision is current master:
https://git.yoctoproject.org/poky/commit/?h=master&id=3e50e45917831d9da2d86e69fb908a1a78483b62

This means that 3.12.1 should fix this problem, and that release is supposedly "imminent": 3.12.0 was released on 2023-10-02 and subsequent bugfix releases are scheduled every two months.

#11 lore raw
In reply to Böszörményi Zoltán
Quoted 11 lines · Zoltan Boszormenyi via lists.openembedded.org

So it's not just Fedora, but Python 3.12 in general.

Googling for problem with threads + fork reveals that it is indeed an issue even in C / POSIX: only the thread executing fork() will become part of the new process, but not the main process starting the thread or any other threads from the original process.

The warning is justified, and apparently the issue not occurring in bitbake with previous Python versions is pure luck.

FYI: Fixed in Fedora 39 if you update to get python-3.12.1-1 or later.

Signature · 2 lines
# Randy MacLeod
# Wind River Linux
#12 lore raw
Quoted 11 lines · Randy MacLeod

On 2023-11-28 4:26 a.m., Zoltan Boszormenyi via lists.openembedded.org wrote:

So it's not just Fedora, but Python 3.12 in general.

Googling for problem with threads + fork reveals that it is indeed an issue even in C / POSIX: only the thread executing fork() will become part of the new process, but not the main process starting the thread or any other threads from the original process.

Quoted 6 lines

The warning is justified, and apparently the issue not occurring in bitbake with previous Python versions is pure luck.

FYI: Fixed in Fedora 39 if you update to get python-3.12.1-1 or later.

But this fixes it only for bitbake master, right?

With bitbake 2.0 used in kirkstone I still see bitbake processes still running after the build is finished.

#13 lore raw
Quoted 14 lines · Martin Jansa

On Thu, Dec 21, 2023 at 7:47 PM Randy MacLeod <[email protected]> wrote:

    On 2023-11-28 4:26 a.m., Zoltan Boszormenyi via
    lists.openembedded.org
    <https://urldefense.com/v3/__http://lists.openembedded.org__;!!AjveYdw8EvQ!dmJtTKioa9DzcUCj9Bnd0SAhV_iwzhIPFdUGgYGvfCcHJTBSqwE1j07pX4x-RmoIfylxSh1gB9shlAshIeRlbYI0P5Q$>
    wrote:
Quoted 8 lines

So it's not just Fedora, but Python 3.12 in general.

Googling for problem with threads + fork reveals that it is indeed an issue even in C / POSIX: only the thread executing fork() will become part of the new process, but not the main process starting the thread or any other threads from the original process.

Quoted 8 lines
Quoted 3 lines

The warning is justified, and apparently the issue not occurring in bitbake with previous Python versions is pure luck.

FYI: Fixed in Fedora 39 if you update to get python-3.12.1-1 or later.

But this fixes it only for bitbake master, right?

Even master might still have some problems. I can do a simple build of a package but in doing:

$ bitbake -k world

I see:
NOTE: No reply from server in 30s (for command ping at 17:50:35.510798)

After some time.

I'm not going to try to debug it at this point in the day/year!

Quoted 2 lines

With bitbake 2.0 used in kirkstone I still see bitbake processes still running after the build is finished.

I didn't try kirkstone. Maybe Steve has that on his list of things to do in his spare time! ;-)

../Randy

Signature · 2 lines
# Randy MacLeod
# Wind River Linux
#14 lore raw

For what it's worth, I am not seeing any of these issues (hanging processes after bitbake exit, timeouts on world builds with all meta-oe-layers) with python 3.12 provided by buildtools-tarball. The issues are probably real, but it's also possible python 3.12 per se is not to blame.

It would probably help if you check if what you're seeing occurs with python 3.12 from the tarball, which you can build from this branch:
https://git.yoctoproject.org/poky-contrib/log/?h=akanavin/py3.12
(the log in the web ui isn't updated yet, I just pushed)

Alex

Quoted 57 lines · Randy MacLeod via lists.openembedded.org

On 2023-12-21 1:56 p.m., Martin Jansa wrote:

Quoted 11 lines · Randy MacLeod

On 2023-11-28 4:26 a.m., Zoltan Boszormenyi via lists.openembedded.org wrote:

So it's not just Fedora, but Python 3.12 in general.

Googling for problem with threads + fork reveals that it is indeed an issue even in C / POSIX: only the thread executing fork() will become part of the new process, but not the main process starting the thread or any other threads from the original process.

Quoted 5 lines

The warning is justified, and apparently the issue not occurring in bitbake with previous Python versions is pure luck.

FYI: Fixed in Fedora 39 if you update to get python-3.12.1-1 or later.

But this fixes it only for bitbake master, right?

Even master might still have some problems. I can do a simple build of a package but in doing:

$ bitbake -k world

I see:
NOTE: No reply from server in 30s (for command ping at 17:50:35.510798)

After some time.

I'm not going to try to debug it at this point in the day/year!

With bitbake 2.0 used in kirkstone I still see bitbake processes still running after the build is finished.

I didn't try kirkstone. Maybe Steve has that on his list of things to do in his spare time! ;-)

../Randy

--
# Randy MacLeod
# Wind River Linux

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#192854): https://lists.openembedded.org/g/openembedded-core/message/192854
Mute This Topic: https://lists.openembedded.org/mt/102826186/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#15 lore raw

FWIW: I see the processes running after bitbake exit only in kirkstone and only with
PRSERV_HOST = "localhost:0"
if I drop PRSERV_HOST then all 3 bitbake-server processes exist together with bitbake itself.

luneos-kirkstone $ rm -f bitbake-cookerdaemon.log
luneos-kirkstone $ ps aux | grep luneos-kirkstone
martin 4184043 0.0 0.0 9124 2048 pts/4 S+ 16:47 0:00 grep
--colour=auto luneos-kirkstone
luneos-kirkstone $ bitbake -k zlib-native
NOTE: Started PRServer with DBfile:
/OE/build/luneos-kirkstone/cache/prserv.sqlite3, Address: 127.0.0.1:35811,
PID: 4184924
Loading cache: 100% |##########################################################################################################################################################################################################|
Time: 0:00:04
Loaded 4570 entries from dependency cache.
...
Sstate summary: Wanted 10 Local 10 Mirrors 0 Missed 0 Current 11 (100%
match, 100% complete)
NOTE: Executing Tasks
NOTE: Tasks Summary: Attempted 92 tasks of which 92 didn't need to be rerun
and all succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 15 seconds
luneos-kirkstone $ ps aux | grep luneos-kirkstone
martin 51324 0.0 0.0 9124 2048 pts/4 S+ 16:54 0:00 grep
--colour=auto luneos-kirkstone
martin 4184584 8.9 0.4 668196 536892 ? Sl 16:47 0:37
bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
/OE/build/luneos-kirkstone/bitbake.lock
/OE/build/luneos-kirkstone/bitbake.sock 0 None 0
martin 4184924 0.0 0.1 340280 196428 ? S 16:47 0:00
bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
/OE/build/luneos-kirkstone/bitbake.lock
/OE/build/luneos-kirkstone/bitbake.sock 0 None 0
martin 4184927 0.0 0.1 340328 194440 ? S 16:47 0:00
bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
/OE/build/luneos-kirkstone/bitbake.lock
/OE/build/luneos-kirkstone/bitbake.sock 0 None 0

luneos-kirkstone $ cat bitbake-cookerdaemon.log
4184584 16:47:38.367593 --- Starting bitbake server pid 4184584 at

2024-01-05 16:47:38.367526 ---

4184584 16:47:38.370315 Started bitbake server pid 4184584
4184584 16:47:38.370831 Entering server connection loop
4184584 16:47:38.372043 Accepting [<socket.socket fd=6, family=1, type=1,
proto=0, laddr=bitbake.sock>] ([])
4184584 16:47:38.376431 Processing Client
4184584 16:47:38.376633 Connecting Client
4184584 16:47:38.377632 Running command ['setFeatures', [2, 1]]
4184584 16:47:38.379355 Command Completed
4184584 16:47:38.379868 Running command ['updateConfig', {'halt': False,
'force': False, 'invalidate_stamp': None, 'dry_run': False,
'dump_signatures': [], 'extra_assume_provided': [], 'profile': False,
'prefile': [], 'postfile': [], 'server_timeout': None, 'nosetscene': False,
'setsceneonly': False, 'skipsetscene': False, 'runall': None, 'runonly':
None, 'writeeventlog': None, 'build_verbose_shell': False,
'build_verbose_stdout': False, 'default_loglevel': 20, 'debug_domains':
{}}, {'SHELL': '/bin/bash', 'BB_ENV_PASSTHROUGH_ADDITIONS': 'MACHINE DISTRO
TCMODE TCLIBC http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY no_proxy
SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS
PARALLEL_MAKE GIT_PROXY_COMMAND GIT_PROXY_IGNORE SOCKS5_PASSWD SOCKS5_USER
WEBOS_DISTRO_BUILD_ID PSEUDO_DISABLED PSEUDO_BUILD', 'DISTRO': 'luneos',
'SSH_AUTH_SOCK': '/tmp/ssh-XXXXXX2ZxYBo/agent.1893', 'SSH_AGENT_PID':
'1894', 'PWD': '/OE/build/luneos-kirkstone', 'LOGNAME': 'martin', 'HOME':
'/home/martin-gentoo', 'MACHINE': 'qemux86-64', 'USER': 'martin', 'PATH':
'/OE/build/luneos-kirkstone/openembedded-core/scripts:/OE/build/luneos-kirkstone/bitbake/bin:/home/martin-gentoo/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/17/bin:/opt/android-sdk-update-manager/tools:/opt/android-sdk-update-manager/platform-tools:/etc/eselect/wine/bin:/usr/lib64/opencascade/bin:/var/lib/snapd/snap/bin',
'LC_ALL': 'en_US.UTF-8', 'SESSION_MANAGER':
'local/jama:@/tmp/.ICE-unix/1838,unix/jama:/tmp/.ICE-unix/1838',
'WINDOWID': '35651587', 'COLORTERM': 'truecolor',
'CSF_MDTVTexturesDirectory': '/usr/share/opencascade/resources/Textures',
'MACHINES': 'qemux86-64', 'XDG_CONFIG_DIRS': '/etc/xdg', 'LESS': '-R -M
--shift 5', 'XDG_SESSION_PATH': '/org/freedesktop/DisplayManager/Session1',
'XDG_MENU_PREFIX': 'xfce-', 'JDK_HOME':
'/etc/java-config-2/current-system-vm', 'CONFIG_PROTECT_MASK':
'/etc/sandbox.d /etc/fonts/fonts.conf /etc/gentoo-release /etc/dconf
/etc/ca-certificates.conf /etc/revdep-rebuild', 'TERMCAP':
'SC|screen.xterm-256color|VT 100/ANSI X3.64 virtual
terminal:DO=\\E[%dB:LE=\\E[%dD:RI=\\E[%dC:UP=\\E[%dA:bs:bt=\\E[Z:cd=\\E[J:ce=\\E[K:cl=\\E[H\\E[J:cm=\\E[%i%d;%dH:ct=\\E[3g:do=^J:nd=\\E[C:pt:rc=\\E8:rs=\\Ec:sc=\\E7:st=\\EH:up=\\EM:le=^H:bl=^G:cr=^M:it#8:ho=\\E[H:nw=\\EE:ta=^I:is=\\E)0:li#64:co#238:am:xn:xv:LP:sr=\\EM:al=\\E[L:AL=\\E[%dL:cs=\\E[%i%d;%dr:dl=\\E[M:DL=\\E[%dM:dc=\\E[P:DC=\\E[%dP:im=\\E[4h:ei=\\E[4l:mi:IC=\\E[%d@:ks=\\E[?1h\\E=:ke=\\E[?1l\\E>:vi=\\E[?25l:ve=\\E[34h\\E[?25h:vs=\\E[34l:ti=\\E[?1049h:te=\\E[?1049l:us=\\E[4m:ue=\\E[24m:so=\\E[3m:se=\\E[23m:mb=\\E[5m:md=\\E[1m:mh=\\E[2m:mr=\\E[7m:me=\\E[m:ms:Co#8:pa#64:AF=\\E[3%dm:AB=\\E[4%dm:op=\\E[39;49m:AX:vb=\\Eg:G0:as=\\E(0:ae=\\E(B:ac=\\140\\140aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~..--++,,hhII00:po=\\E[5i:pf=\\E[4i:Z0=\\E[?3h:Z1=\\E[?3l:Km=\\E[<:k0=\\E[10~:k1=\\EOP:k2=\\EOQ:k3=\\EOR:k4=\\EOS:k5=\\E[15~:k6=\\E[17~:k7=\\E[18~:k8=\\E[19~:k9=\\E[20~:k;=\\E[21~:F1=\\E[23~:F2=\\E[24~:kB=\\E[Z:kh=\\E[1~:@1=\\E[1~:kH=\\E[4~:@7=\\E[4~:kN=\\E[6~:kP=\\E[5~:kI=\\E[2~:kD=\\E[3~:ku=\\EOA:kd=\\EOB:kr=\\EOC:kl=\\EOD:km:',
'JAVA_HOME': '/etc/java-config-2/current-system-vm', 'CSF_MIGRATION_TYPES':
'/usr/share/opencascade/resources/StdResource/MigrationSheet.txt',
'WINDOW': '3', 'ANT_HOME': '/usr/share/ant', 'DESKTOP_SESSION': 'xfce',
'CSF_STEPDefaults': '/usr/share/opencascade/resources/XSTEPResource',
'EDITOR': 'vim', 'GTK_MODULES': 'canberra-gtk-module', 'XDG_SEAT': 'seat0',
'CONFIG_PROTECT': '/var/bind /usr/share/gnupg/qualified.txt
/usr/share/easy-rsa /usr/share/config
/usr/lib64/libreoffice/program/sofficerc', 'XDG_SESSION_DESKTOP': 'XFCE',
'XDG_SESSION_TYPE': 'x11', 'PANEL_GDK_CORE_DEVICE_EVENTS': '0', 'MANPATH':
'/etc/java-config-2/current-system-vm/man:/usr/share/gcc-data/x86_64-pc-linux-gnu/13/man:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.41/man:/etc/java-config-2/current-system-vm/man/:/usr/local/share/man:/usr/share/man:/usr/lib/rust/man:/usr/lib/llvm/17/share/man:/etc/eselect/wine/share/man',
'XAUTHORITY': '/tmp/xauth_CEynPY', 'CSF_StandardLiteDefaults':
'/usr/share/opencascade/resources/StdResource', 'OPENCL_PROFILE': 'nvidia',
'MOTD_SHOWN': 'pam', 'GDM_LANG': 'C.UTF-8', 'LANG': 'en_US.UTF-8',
'LS_COLORS':
'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:su=37;41:sg=30;43:ca=00:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.avif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.cfg=00;32:*.conf=00;32:*.diff=00;32:*.doc=00;32:*.ini=00;32:*.log=00;32:*.patch=00;32:*.pdf=00;32:*.ps=00;32:*.tex=00;32:*.txt=00;32:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:*~=00;90:*#=00;90:*.bak=00;90:*.crdownload=00;90:*.dpkg-dist=00;90:*.dpkg-new=00;90:*.dpkg-old=00;90:*.dpkg-tmp=00;90:*.old=00;90:*.orig=00;90:*.part=00;90:*.rej=00;90:*.rpmnew=00;90:*.rpmorig=00;90:*.rpmsave=00;90:*.swp=00;90:*.tmp=00;90:*.ucf-dist=00;90:*.ucf-new=00;90:*.ucf-old=00;90:', 'XDG_CURRENT_DESKTOP': 'XFCE', 'VTE_VERSION': '7402', 'CSF_GraphicShr':
'/usr/lib64/opencascade/libTKOpenGl.so', 'OPENGL_PROFILE': 'xorg-x11',
'XDG_SEAT_PATH': '/org/freedesktop/DisplayManager/Seat0',
'QT_GRAPHICSSYSTEM': 'raster', 'XZ_OPT': '-T0', 'CSF_ShadersDirectory':
'/usr/share/opencascade/resources/Shaders', 'CSF_XmlOcafResource':
'/usr/share/opencascade/resources/XmlOcafResource', 'INFOPATH':
'/usr/share/gcc-data/x86_64-pc-linux-gnu/13/info:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.41/info:/usr/share/autoconf-2.72/info:/usr/share/autoconf-2.13/info:/usr/share/automake-1.16.5/info:/usr/share/info',
'MOZ_GMP_PATH':
'/usr/lib64/nsbrowser/plugins/gmp-gmpopenh264/system-installed',
'CSF_SHMessage': '/usr/share/opencascade/resources/SHMessage', 'JAVAC':
'/etc/java-config-2/current-system-vm/bin/javac', 'XDG_SESSION_CLASS':
'user', 'VBOX_SDK_PATH': '/usr/lib64/virtualbox/sdk', 'ANDROID_HOME':
'/opt/android-sdk-update-manager', 'TERM': 'screen.xterm-256color',
'VBOX_INSTALL_PATH': '/usr/lib64/virtualbox', 'LESSOPEN': '|lesspipe %s',
'ANDROID_SWT': '/usr/share/swt-3.7/lib', 'CSF_StandardDefaults':
'/usr/share/opencascade/resources/StdResource', 'CSF_IGESDefaults':
'/usr/share/opencascade/resources/XSTEPResource', 'MANPAGER': 'manpager',
'NPM_CONFIG_GLOBALCONFIG': '/etc/npm/npmrc', 'DISPLAY': ':0.0',
'CSF_XCAFDefaults': '/usr/share/opencascade/resources/StdResource',
'SHLVL': '2', 'PAGER': '/usr/bin/less', 'LEX': 'flex', 'XDG_VTNR': '2',
'CSF_PluginDefaults': '/usr/share/opencascade/resources/StdResource',
'XDG_SESSION_ID': '2', 'CASROOT': '/usr', 'XDG_RUNTIME_DIR':
'/run/user/1026', 'GCC_SPECS': '', 'GSETTINGS_BACKEND': 'dconf', 'PS1':
'\\[\\033[01;32m\\]OE $MACHINE@$DISTRO\\[\\033[01;34m\\] \\w
\\$\\[\\033[00m\\] ', 'CSF_XSMessage':
'/usr/share/opencascade/resources/XSMessage', 'XDG_DATA_DIRS':
'/usr/local/share:/usr/share:/etc/eselect/wine/share:/var/lib/snapd/desktop',
'CSF_UnitsDefinition':
'/usr/share/opencascade/resources/UnitsAPI/Units.dat', 'STY': '2067.x',
'VBOX_APP_HOME': '/usr/lib64/virtualbox', 'DBUS_SESSION_BUS_ADDRESS':
'unix:path=/run/user/1026/bus', 'INITPWD': '/OE/build/luneos-kirkstone',
'MAIL': '/var/mail/martin', 'OLDPWD': '/home/martin-gentoo', '_':
'/OE/build/luneos-kirkstone/bitbake/bin/bitbake'},
['/OE/build/luneos-kirkstone/bitbake/bin/bitbake', '-k', 'zlib-native']]
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228:
DeprecationWarning: ast.Str is deprecated and will be removed in Python
3.14; use ast.Constant instead
if isinstance(node.args[0], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229:
DeprecationWarning: Attribute s is deprecated and will be removed in Python
3.14; use value instead
varname = node.args[0].s
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:230:
DeprecationWarning: ast.Str is deprecated and will be removed in Python
3.14; use ast.Constant instead
if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:233:
DeprecationWarning: Attribute s is deprecated and will be removed in Python
3.14; use value instead
self.contains[varname].add(node.args[1].s)
/usr/lib/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning:
This process (pid=4184584) is multi-threaded, use of fork() may lead to
deadlocks in the child.
self.pid = os.fork()
4184584 16:47:45.108449 Command Completed
4184584 16:47:45.112267 Running command ['getVariable', 'BBINCLUDELOGS']
4184584 16:47:45.113080 Command Completed
4184584 16:47:45.113399 Running command ['getVariable',
'BBINCLUDELOGS_LINES']
4184584 16:47:45.113925 Command Completed
4184584 16:47:45.114218 Running command ['getSetVariable', 'BB_CONSOLELOG']
4184584 16:47:45.114956 Command Completed
4184584 16:47:45.115441 Running command ['getSetVariable', 'BB_LOGCONFIG']
4184584 16:47:45.116590 Command Completed
4184584 16:47:45.122232 Running command ['getUIHandlerNum']
4184584 16:47:45.122805 Command Completed
4184584 16:47:45.123194 Running command ['setEventMask', 1, 20,
{'BitBake.SigGen.HashEquiv': 19, 'BitBake.RunQueue.HashEquiv': 19},
['bb.runqueue.runQueueExitWait', 'bb.event.LogExecTTY',
'logging.LogRecord', 'bb.build.TaskFailed', 'bb.build.TaskBase',
'bb.event.ParseStarted', 'bb.event.ParseProgress',
'bb.event.ParseCompleted', 'bb.event.CacheLoadStarted',
'bb.event.CacheLoadProgress', 'bb.event.CacheLoadCompleted',
'bb.command.CommandFailed', 'bb.command.CommandExit',
'bb.command.CommandCompleted', 'bb.cooker.CookerExit',
'bb.event.MultipleProviders', 'bb.event.NoProvider',
'bb.runqueue.sceneQueueTaskStarted', 'bb.runqueue.runQueueTaskStarted',
'bb.runqueue.runQueueTaskFailed', 'bb.runqueue.sceneQueueTaskFailed',
'bb.event.BuildBase', 'bb.build.TaskStarted', 'bb.build.TaskSucceeded',
'bb.build.TaskFailedSilent', 'bb.build.TaskProgress',
'bb.event.ProcessStarted', 'bb.event.ProcessProgress',
'bb.event.ProcessFinished']]
4184584 16:47:45.123558 Command Completed
4184584 16:47:45.123863 Running command ['getVariable', 'BB_DEFAULT_TASK']
4184584 16:47:45.124228 Command Completed
4184584 16:47:45.124528 Running command ['setConfig', 'cmd', 'build']
4184584 16:47:45.124858 Command Completed
4184584 16:47:45.125140 Running command ['buildTargets', ['zlib-native'],
'build']
4184584 16:47:45.125294 Command Completed
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228:
DeprecationWarning: ast.Str is deprecated and will be removed in Python
3.14; use ast.Constant instead
if isinstance(node.args[0], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229:
DeprecationWarning: Attribute s is deprecated and will be removed in Python
3.14; use value instead
varname = node.args[0].s
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:230:
DeprecationWarning: ast.Str is deprecated and will be removed in Python
3.14; use ast.Constant instead
if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:233:
DeprecationWarning: Attribute s is deprecated and will be removed in Python
3.14; use value instead
self.contains[varname].add(node.args[1].s)
/usr/lib/python3.12/asyncio/streams.py:413: ResourceWarning: unclosed
<StreamWriter transport=<_SelectorSocketTransport closing fd=18 read=idle
write=<idle, bufsize=0>> reader=<StreamReader
transport=<_SelectorSocketTransport closing fd=18 read=idle write=<idle,
bufsize=0>>>>
warnings.warn(f"unclosed {self!r}", ResourceWarning)
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228:
DeprecationWarning: ast.Str is deprecated and will be removed in Python
3.14; use ast.Constant instead
if isinstance(node.args[0], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229:
DeprecationWarning: Attribute s is deprecated and will be removed in Python
3.14; use value instead
varname = node.args[0].s
4184584 16:48:37.909973 Processing Client
4184584 16:48:37.910125 Disconnecting Client
4184584 16:48:37.910729 No timeout, exiting.
4184584 16:48:38.011341 Exiting

It is reproducible with nanbield as well and fixed in scarthgap, will bisect it when time allows.

Regards,

Quoted 79 lines · Alexander Kanavin

For what it's worth, I am not seeing any of these issues (hanging processes after bitbake exit, timeouts on world builds with all meta-oe-layers) with python 3.12 provided by buildtools-tarball. The issues are probably real, but it's also possible python 3.12 per se is not to blame.

It would probably help if you check if what you're seeing occurs with python 3.12 from the tarball, which you can build from this branch:
https://git.yoctoproject.org/poky-contrib/log/?h=akanavin/py3.12
(the log in the web ui isn't updated yet, I just pushed)

Alex

Quoted 4 lines · Randy MacLeod via lists.openembedded.org

On 2023-12-21 1:56 p.m., Martin Jansa wrote:

On Thu, Dec 21, 2023 at 7:47 PM Randy MacLeod <

Quoted 2 lines · [email protected]> wrote:
Quoted 2 lines

On 2023-11-28 4:26 a.m., Zoltan Boszormenyi via lists.openembedded.org

Quoted 26 lines · wrote:
Quoted 9 lines

So it's not just Fedora, but Python 3.12 in general.

Googling for problem with threads + fork reveals that it is indeed an issue even in C / POSIX: only the thread executing fork() will become part of the new process, but not the main process starting the thread or any other threads from the original process.

Quoted 5 lines

The warning is justified, and apparently the issue not occurring in bitbake with previous Python versions is pure luck.

FYI: Fixed in Fedora 39 if you update to get python-3.12.1-1 or later.

But this fixes it only for bitbake master, right?

Even master might still have some problems. I can do a simple build of a package but in doing:

$ bitbake -k world

I see:
NOTE: No reply from server in 30s (for command ping at 17:50:35.510798)

Quoted 6 lines

After some time.

I'm not going to try to debug it at this point in the day/year!

With bitbake 2.0 used in kirkstone I still see bitbake processes still

running after the build is finished.

Quoted 2 lines

I didn't try kirkstone. Maybe Steve has that on his list of things to do

in his spare time! ;-)

Quoted 4 lines

../Randy

Quoted 8 lines

--
# Randy MacLeod
# Wind River Linux

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#192854):

https://lists.openembedded.org/g/openembedded-core/message/192854

Quoted 3 lines

Mute This Topic: https://lists.openembedded.org/mt/102826186/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [

[email protected]]

Quoted 2 lines

-=-=-=-=-=-=-=-=-=-=-=-

A message that is not in the archive (sent off-list or not yet received): [email protected]

#17 lore raw
Quoted 223 lines · Martin Jansa via lists.openembedded.org

FWIW: I see the processes running after bitbake exit only in kirkstone and only with
PRSERV_HOST = "localhost:0"
if I drop PRSERV_HOST then all 3 bitbake-server processes exist together with bitbake itself.

luneos-kirkstone $ rm -f bitbake-cookerdaemon.log
luneos-kirkstone $ ps aux | grep luneos-kirkstone
martin 4184043 0.0 0.0 9124 2048 pts/4 S+ 16:47 0:00 grep
--colour=auto luneos-kirkstone
luneos-kirkstone $ bitbake -k zlib-native
NOTE: Started PRServer with DBfile:
/OE/build/luneos-kirkstone/cache/prserv.sqlite3, Address: 127.0.0.1:35811,
PID: 4184924
Loading cache: 100% |##########################################################################################################################################################################################################|
Time: 0:00:04
Loaded 4570 entries from dependency cache.
...
Sstate summary: Wanted 10 Local 10 Mirrors 0 Missed 0 Current 11 (100%
match, 100% complete)
NOTE: Executing Tasks
NOTE: Tasks Summary: Attempted 92 tasks of which 92 didn't need to be
rerun and all succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 15 seconds
luneos-kirkstone $ ps aux | grep luneos-kirkstone
martin 51324 0.0 0.0 9124 2048 pts/4 S+ 16:54 0:00 grep
--colour=auto luneos-kirkstone
martin 4184584 8.9 0.4 668196 536892 ? Sl 16:47 0:37
bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
/OE/build/luneos-kirkstone/bitbake.lock
/OE/build/luneos-kirkstone/bitbake.sock 0 None 0
martin 4184924 0.0 0.1 340280 196428 ? S 16:47 0:00
bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
/OE/build/luneos-kirkstone/bitbake.lock
/OE/build/luneos-kirkstone/bitbake.sock 0 None 0
martin 4184927 0.0 0.1 340328 194440 ? S 16:47 0:00
bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
/OE/build/luneos-kirkstone/bitbake.lock
/OE/build/luneos-kirkstone/bitbake.sock 0 None 0

luneos-kirkstone $ cat bitbake-cookerdaemon.log
4184584 16:47:38.367593 --- Starting bitbake server pid 4184584 at

2024-01-05 16:47:38.367526 ---

4184584 16:47:38.370315 Started bitbake server pid 4184584
4184584 16:47:38.370831 Entering server connection loop
4184584 16:47:38.372043 Accepting [<socket.socket fd=6, family=1, type=1,
proto=0, laddr=bitbake.sock>] ([])
4184584 16:47:38.376431 Processing Client
4184584 16:47:38.376633 Connecting Client
4184584 16:47:38.377632 Running command ['setFeatures', [2, 1]]
4184584 16:47:38.379355 Command Completed
4184584 16:47:38.379868 Running command ['updateConfig', {'halt': False,
'force': False, 'invalidate_stamp': None, 'dry_run': False,
'dump_signatures': [], 'extra_assume_provided': [], 'profile': False,
'prefile': [], 'postfile': [], 'server_timeout': None, 'nosetscene': False,
'setsceneonly': False, 'skipsetscene': False, 'runall': None, 'runonly':
None, 'writeeventlog': None, 'build_verbose_shell': False,
'build_verbose_stdout': False, 'default_loglevel': 20, 'debug_domains':
{}}, {'SHELL': '/bin/bash', 'BB_ENV_PASSTHROUGH_ADDITIONS': 'MACHINE DISTRO
TCMODE TCLIBC http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY no_proxy
SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS
PARALLEL_MAKE GIT_PROXY_COMMAND GIT_PROXY_IGNORE SOCKS5_PASSWD SOCKS5_USER
WEBOS_DISTRO_BUILD_ID PSEUDO_DISABLED PSEUDO_BUILD', 'DISTRO': 'luneos',
'SSH_AUTH_SOCK': '/tmp/ssh-XXXXXX2ZxYBo/agent.1893', 'SSH_AGENT_PID':
'1894', 'PWD': '/OE/build/luneos-kirkstone', 'LOGNAME': 'martin', 'HOME':
'/home/martin-gentoo', 'MACHINE': 'qemux86-64', 'USER': 'martin', 'PATH':
'/OE/build/luneos-kirkstone/openembedded-core/scripts:/OE/build/luneos-kirkstone/bitbake/bin:/home/martin-gentoo/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/17/bin:/opt/android-sdk-update-manager/tools:/opt/android-sdk-update-manager/platform-tools:/etc/eselect/wine/bin:/usr/lib64/opencascade/bin:/var/lib/snapd/snap/bin',
'LC_ALL': 'en_US.UTF-8', 'SESSION_MANAGER': 'local/jama:@/tmp/.ICE-unix/1838,unix/jama:/tmp/.ICE-unix/1838',
'WINDOWID': '35651587', 'COLORTERM': 'truecolor',
'CSF_MDTVTexturesDirectory': '/usr/share/opencascade/resources/Textures',
'MACHINES': 'qemux86-64', 'XDG_CONFIG_DIRS': '/etc/xdg', 'LESS': '-R -M
--shift 5', 'XDG_SESSION_PATH': '/org/freedesktop/DisplayManager/Session1',
'XDG_MENU_PREFIX': 'xfce-', 'JDK_HOME':
'/etc/java-config-2/current-system-vm', 'CONFIG_PROTECT_MASK':
'/etc/sandbox.d /etc/fonts/fonts.conf /etc/gentoo-release /etc/dconf
/etc/ca-certificates.conf /etc/revdep-rebuild', 'TERMCAP':
'SC|screen.xterm-256color|VT 100/ANSI X3.64 virtual
terminal:DO=\\E[%dB:LE=\\E[%dD:RI=\\E[%dC:UP=\\E[%dA:bs:bt=\\E[Z:cd=\\E[J:ce=\\E[K:cl=\\E[H\\E[J:cm=\\E[%i%d;%dH:ct=\\E[3g:do=^J:nd=\\E[C:pt:rc=\\E8:rs=\\Ec:sc=\\E7:st=\\EH:up=\\EM:le=^H:bl=^G:cr=^M:it#8:ho=\\E[H:nw=\\EE:ta=^I:is=\\E)0:li#64:co#238:am:xn:xv:LP:sr=\\EM:al=\\E[L:AL=\\E[%dL:cs=\\E[%i%d;%dr:dl=\\E[M:DL=\\E[%dM:dc=\\E[P:DC=\\E[%dP:im=\\E[4h:ei=\\E[4l:mi:IC=\\E[%d@:ks=\\E[?1h\\E=:ke=\\E[?1l\\E>:vi=\\E[?25l:ve=\\E[34h\\E[?25h:vs=\\E[34l:ti=\\E[?1049h:te=\\E[?1049l:us=\\E[4m:ue=\\E[24m:so=\\E[3m:se=\\E[23m:mb=\\E[5m:md=\\E[1m:mh=\\E[2m:mr=\\E[7m:me=\\E[m:ms:Co#8:pa#64:AF=\\E[3%dm:AB=\\E[4%dm:op=\\E[39;49m:AX:vb=\\Eg:G0:as=\\E(0:ae=\\E(B:ac=\\140\\140aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~..--++,,hhII00:po=\\E[5i:pf=\\E[4i:Z0=\\E[?3h:Z1=\\E[?3l:Km=\\E[<:k0=\\E[10~:k1=\\EOP:k2=\\EOQ:k3=\\EOR:k4=\\EOS:k5=\\E[15~:k6=\\E[17~:k7=\\E[18~:k8=\\E[19~:k9=\\E[20~:k;=\\E[21~:F1=\\E[23~:F2=\\E[24~:kB=\\E[Z:kh=\\E[1~:@1=\\E[1~:kH=\\E[4~:@7=\\E[4~:kN=\\E[6~:kP=\\E[5~:kI=\\E[2~:kD=\\E[3~:ku=\\EOA:kd=\\EOB:kr=\\EOC:kl=\\EOD:km:',
'JAVA_HOME': '/etc/java-config-2/current-system-vm', 'CSF_MIGRATION_TYPES':
'/usr/share/opencascade/resources/StdResource/MigrationSheet.txt',
'WINDOW': '3', 'ANT_HOME': '/usr/share/ant', 'DESKTOP_SESSION': 'xfce',
'CSF_STEPDefaults': '/usr/share/opencascade/resources/XSTEPResource',
'EDITOR': 'vim', 'GTK_MODULES': 'canberra-gtk-module', 'XDG_SEAT': 'seat0',
'CONFIG_PROTECT': '/var/bind /usr/share/gnupg/qualified.txt
/usr/share/easy-rsa /usr/share/config
/usr/lib64/libreoffice/program/sofficerc', 'XDG_SESSION_DESKTOP': 'XFCE',
'XDG_SESSION_TYPE': 'x11', 'PANEL_GDK_CORE_DEVICE_EVENTS': '0', 'MANPATH':
'/etc/java-config-2/current-system-vm/man:/usr/share/gcc-data/x86_64-pc-linux-gnu/13/man:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.41/man:/etc/java-config-2/current-system-vm/man/:/usr/local/share/man:/usr/share/man:/usr/lib/rust/man:/usr/lib/llvm/17/share/man:/etc/eselect/wine/share/man',
'XAUTHORITY': '/tmp/xauth_CEynPY', 'CSF_StandardLiteDefaults':
'/usr/share/opencascade/resources/StdResource', 'OPENCL_PROFILE': 'nvidia',
'MOTD_SHOWN': 'pam', 'GDM_LANG': 'C.UTF-8', 'LANG': 'en_US.UTF-8',
'LS_COLORS':
'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:su=37;41:sg=30;43:ca=00:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.avif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.cfg=00;32:*.conf=00;32:*.diff=00;32:*.doc=00;32:*.ini=00;32:*.log=00;32:*.patch=00;32:*.pdf=00;32:*.ps=00;32:*.tex=00;32:*.txt=00;32:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:*~=00;90:*#=00;90:*.bak=00;90:*.crdownload=00;90:*.dpkg-dist=00;90:*.dpkg-new=00;90:*.dpkg-old=00;90:*.dpkg-tmp=00;90:*.old=00;90:*.orig=00;90:*.part=00;90:*.rej=00;90:*.rpmnew=00;90:*.rpmorig=00;90:*.rpmsave=00;90:*.swp=00;90:*.tmp=00;90:*.ucf-dist=00;90:*.ucf-new=00;90:*.ucf-old=00;90:', 'XDG_CURRENT_DESKTOP': 'XFCE', 'VTE_VERSION': '7402', 'CSF_GraphicShr':
'/usr/lib64/opencascade/libTKOpenGl.so', 'OPENGL_PROFILE': 'xorg-x11',
'XDG_SEAT_PATH': '/org/freedesktop/DisplayManager/Seat0',
'QT_GRAPHICSSYSTEM': 'raster', 'XZ_OPT': '-T0', 'CSF_ShadersDirectory':
'/usr/share/opencascade/resources/Shaders', 'CSF_XmlOcafResource':
'/usr/share/opencascade/resources/XmlOcafResource', 'INFOPATH':
'/usr/share/gcc-data/x86_64-pc-linux-gnu/13/info:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.41/info:/usr/share/autoconf-2.72/info:/usr/share/autoconf-2.13/info:/usr/share/automake-1.16.5/info:/usr/share/info',
'MOZ_GMP_PATH':
'/usr/lib64/nsbrowser/plugins/gmp-gmpopenh264/system-installed',
'CSF_SHMessage': '/usr/share/opencascade/resources/SHMessage', 'JAVAC':
'/etc/java-config-2/current-system-vm/bin/javac', 'XDG_SESSION_CLASS':
'user', 'VBOX_SDK_PATH': '/usr/lib64/virtualbox/sdk', 'ANDROID_HOME':
'/opt/android-sdk-update-manager', 'TERM': 'screen.xterm-256color',
'VBOX_INSTALL_PATH': '/usr/lib64/virtualbox', 'LESSOPEN': '|lesspipe %s',
'ANDROID_SWT': '/usr/share/swt-3.7/lib', 'CSF_StandardDefaults':
'/usr/share/opencascade/resources/StdResource', 'CSF_IGESDefaults':
'/usr/share/opencascade/resources/XSTEPResource', 'MANPAGER': 'manpager',
'NPM_CONFIG_GLOBALCONFIG': '/etc/npm/npmrc', 'DISPLAY': ':0.0',
'CSF_XCAFDefaults': '/usr/share/opencascade/resources/StdResource',
'SHLVL': '2', 'PAGER': '/usr/bin/less', 'LEX': 'flex', 'XDG_VTNR': '2',
'CSF_PluginDefaults': '/usr/share/opencascade/resources/StdResource',
'XDG_SESSION_ID': '2', 'CASROOT': '/usr', 'XDG_RUNTIME_DIR':
'/run/user/1026', 'GCC_SPECS': '', 'GSETTINGS_BACKEND': 'dconf', 'PS1':
'\\[\\033[01;32m\\]OE $MACHINE@$DISTRO\\[\\033[01;34m\\] \\w
\\$\\[\\033[00m\\] ', 'CSF_XSMessage':
'/usr/share/opencascade/resources/XSMessage', 'XDG_DATA_DIRS':
'/usr/local/share:/usr/share:/etc/eselect/wine/share:/var/lib/snapd/desktop',
'CSF_UnitsDefinition':
'/usr/share/opencascade/resources/UnitsAPI/Units.dat', 'STY': '2067.x',
'VBOX_APP_HOME': '/usr/lib64/virtualbox', 'DBUS_SESSION_BUS_ADDRESS':
'unix:path=/run/user/1026/bus', 'INITPWD': '/OE/build/luneos-kirkstone',
'MAIL': '/var/mail/martin', 'OLDPWD': '/home/martin-gentoo', '_':
'/OE/build/luneos-kirkstone/bitbake/bin/bitbake'},
['/OE/build/luneos-kirkstone/bitbake/bin/bitbake', '-k', 'zlib-native']]
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228:
DeprecationWarning: ast.Str is deprecated and will be removed in Python
3.14; use ast.Constant instead
if isinstance(node.args[0], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229:
DeprecationWarning: Attribute s is deprecated and will be removed in Python
3.14; use value instead
varname = node.args[0].s
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:230:
DeprecationWarning: ast.Str is deprecated and will be removed in Python
3.14; use ast.Constant instead
if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:233:
DeprecationWarning: Attribute s is deprecated and will be removed in Python
3.14; use value instead
self.contains[varname].add(node.args[1].s)
/usr/lib/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning:
This process (pid=4184584) is multi-threaded, use of fork() may lead to
deadlocks in the child.
self.pid = os.fork()
4184584 16:47:45.108449 Command Completed
4184584 16:47:45.112267 Running command ['getVariable', 'BBINCLUDELOGS']
4184584 16:47:45.113080 Command Completed
4184584 16:47:45.113399 Running command ['getVariable',
'BBINCLUDELOGS_LINES']
4184584 16:47:45.113925 Command Completed
4184584 16:47:45.114218 Running command ['getSetVariable', 'BB_CONSOLELOG']
4184584 16:47:45.114956 Command Completed
4184584 16:47:45.115441 Running command ['getSetVariable', 'BB_LOGCONFIG']
4184584 16:47:45.116590 Command Completed
4184584 16:47:45.122232 Running command ['getUIHandlerNum']
4184584 16:47:45.122805 Command Completed
4184584 16:47:45.123194 Running command ['setEventMask', 1, 20,
{'BitBake.SigGen.HashEquiv': 19, 'BitBake.RunQueue.HashEquiv': 19},
['bb.runqueue.runQueueExitWait', 'bb.event.LogExecTTY',
'logging.LogRecord', 'bb.build.TaskFailed', 'bb.build.TaskBase',
'bb.event.ParseStarted', 'bb.event.ParseProgress',
'bb.event.ParseCompleted', 'bb.event.CacheLoadStarted',
'bb.event.CacheLoadProgress', 'bb.event.CacheLoadCompleted',
'bb.command.CommandFailed', 'bb.command.CommandExit',
'bb.command.CommandCompleted', 'bb.cooker.CookerExit',
'bb.event.MultipleProviders', 'bb.event.NoProvider',
'bb.runqueue.sceneQueueTaskStarted', 'bb.runqueue.runQueueTaskStarted',
'bb.runqueue.runQueueTaskFailed', 'bb.runqueue.sceneQueueTaskFailed',
'bb.event.BuildBase', 'bb.build.TaskStarted', 'bb.build.TaskSucceeded',
'bb.build.TaskFailedSilent', 'bb.build.TaskProgress',
'bb.event.ProcessStarted', 'bb.event.ProcessProgress',
'bb.event.ProcessFinished']]
4184584 16:47:45.123558 Command Completed
4184584 16:47:45.123863 Running command ['getVariable', 'BB_DEFAULT_TASK']
4184584 16:47:45.124228 Command Completed
4184584 16:47:45.124528 Running command ['setConfig', 'cmd', 'build']
4184584 16:47:45.124858 Command Completed
4184584 16:47:45.125140 Running command ['buildTargets', ['zlib-native'],
'build']
4184584 16:47:45.125294 Command Completed
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228:
DeprecationWarning: ast.Str is deprecated and will be removed in Python
3.14; use ast.Constant instead
if isinstance(node.args[0], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229:
DeprecationWarning: Attribute s is deprecated and will be removed in Python
3.14; use value instead
varname = node.args[0].s
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:230:
DeprecationWarning: ast.Str is deprecated and will be removed in Python
3.14; use ast.Constant instead
if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:233:
DeprecationWarning: Attribute s is deprecated and will be removed in Python
3.14; use value instead
self.contains[varname].add(node.args[1].s)
/usr/lib/python3.12/asyncio/streams.py:413: ResourceWarning: unclosed
<StreamWriter transport=<_SelectorSocketTransport closing fd=18 read=idle
write=<idle, bufsize=0>> reader=<StreamReader
transport=<_SelectorSocketTransport closing fd=18 read=idle write=<idle,
bufsize=0>>>>
warnings.warn(f"unclosed {self!r}", ResourceWarning)
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228:
DeprecationWarning: ast.Str is deprecated and will be removed in Python
3.14; use ast.Constant instead
if isinstance(node.args[0], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229:
DeprecationWarning: Attribute s is deprecated and will be removed in Python
3.14; use value instead
varname = node.args[0].s
4184584 16:48:37.909973 Processing Client
4184584 16:48:37.910125 Disconnecting Client
4184584 16:48:37.910729 No timeout, exiting.
4184584 16:48:38.011341 Exiting

Quoted 3 lines

It is reproducible with nanbield as well and fixed in scarthgap, will bisect it when time allows.

bisect was simpler than expected as nanbield is still compatible with newer bitbake and there weren't that many changes.

nanbield as well as kirkstone can be fixed by cherry-picking:
https://git.openembedded.org/bitbake/commit/?id=d01d684a0 asyncrpc: Add context manager API

+ Joshua if he sees some risks in cherry-picking this to stable releases.

Cheers

#18 lore raw
Quoted 87 lines · Martin Jansa
Quoted 82 lines · Martin Jansa via lists.openembedded.org

FWIW: I see the processes running after bitbake exit only in kirkstone and only with PRSERV_HOST = "localhost:0"
if I drop PRSERV_HOST then all 3 bitbake-server processes exist together with bitbake itself.

luneos-kirkstone $ rm -f bitbake-cookerdaemon.log
luneos-kirkstone $ ps aux | grep luneos-kirkstone
martin 4184043 0.0 0.0 9124 2048 pts/4 S+ 16:47 0:00 grep --colour=auto luneos-kirkstone
luneos-kirkstone $ bitbake -k zlib-native
NOTE: Started PRServer with DBfile: /OE/build/luneos-kirkstone/cache/prserv.sqlite3, Address: 127.0.0.1:35811, PID: 4184924
Loading cache: 100% |##########################################################################################################################################################################################################| Time: 0:00:04 Loaded 4570 entries from dependency cache.
...
Sstate summary: Wanted 10 Local 10 Mirrors 0 Missed 0 Current 11 (100% match, 100% complete)
NOTE: Executing Tasks
NOTE: Tasks Summary: Attempted 92 tasks of which 92 didn't need to be rerun and all succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 15 seconds
luneos-kirkstone $ ps aux | grep luneos-kirkstone

martin     51324  0.0  0.0   9124  2048 pts/4    S+   16:54   0:00 grep --colour=auto luneos-kirkstone
martin   4184584  8.9  0.4 668196 536892 ?       Sl   16:47   0:37 bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log /OE/build/luneos-kirkstone/bitbake.lock /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
martin   4184924  0.0  0.1 340280 196428 ?       S    16:47   0:00 bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log /OE/build/luneos-kirkstone/bitbake.lock /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
martin   4184927  0.0  0.1 340328 194440 ?       S    16:47   0:00 bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log /OE/build/luneos-kirkstone/bitbake.lock /OE/build/luneos-kirkstone/bitbake.sock 0 None 0

luneos-kirkstone $ cat bitbake-cookerdaemon.log
4184584 16:47:38.367593 --- Starting bitbake server pid 4184584 at 2024-01-05 16:47:38.367526 ---
4184584 16:47:38.370315 Started bitbake server pid 4184584
4184584 16:47:38.370831 Entering server connection loop
4184584 16:47:38.372043 Accepting [<socket.socket fd=6, family=1, type=1, proto=0, laddr=bitbake.sock>] ([])
4184584 16:47:38.376431 Processing Client
4184584 16:47:38.376633 Connecting Client
4184584 16:47:38.377632 Running command ['setFeatures', [2, 1]]
4184584 16:47:38.379355 Command Completed
4184584 16:47:38.379868 Running command ['updateConfig', {'halt': False, 'force': False, 'invalidate_stamp': None, 'dry_run': False, 'dump_signatures': [], 'extra_assume_provided': [], 'profile': False, 'prefile': [], 'postfile': [], 'server_timeout': None, 'nosetscene': False, 'setsceneonly': False, 'skipsetscene': False, 'runall': None, 'runonly': None, 'writeeventlog': None, 'build_verbose_shell': False, 'build_verbose_stdout': False, 'default_loglevel': 20, 'debug_domains': {}}, {'SHELL': '/bin/bash', 'BB_ENV_PASSTHROUGH_ADDITIONS': 'MACHINE DISTRO TCMODE TCLIBC http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS PARALLEL_MAKE GIT_PROXY_COMMAND GIT_PROXY_IGNORE SOCKS5_PASSWD SOCKS5_USER WEBOS_DISTRO_BUILD_ID PSEUDO_DISABLED PSEUDO_BUILD', 'DISTRO': 'luneos', 'SSH_AUTH_SOCK': '/tmp/ssh-XXXXXX2ZxYBo/agent.1893', 'SSH_AGENT_PID': '1894', 'PWD': '/OE/build/luneos-kirkstone', 'LOGNAME': 'martin', 'HOME': '/home/martin-gentoo', 'MACHINE': 'qemux86-64', 'USER': 'martin', 'PATH': '/OE/build/luneos-kirkstone/openembedded-core/scripts:/OE/build/luneos-kirkstone/bitbake/bin:/home/martin-gentoo/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/17/bin:/opt/android-sdk-update-manager/tools:/opt/android-sdk-update-manager/platform-tools:/etc/eselect/wine/bin:/usr/lib64/opencascade/bin:/var/lib/snapd/snap/bin', 'LC_ALL': 'en_US.UTF-8', 'SESSION_MANAGER': 'local/jama:@/tmp/.ICE-unix/1838,unix/jama:/tmp/.ICE-unix/1838', 'WINDOWID': '35651587', 'COLORTERM': 'truecolor', 'CSF_MDTVTexturesDirectory': '/usr/share/opencascade/resources/Textures', 'MACHINES': 'qemux86-64', 'XDG_CONFIG_DIRS': '/etc/xdg', 'LESS': '-R -M --shift 5', 'XDG_SESSION_PATH': '/org/freedesktop/DisplayManager/Session1', 'XDG_MENU_PREFIX': 'xfce-', 'JDK_HOME': '/etc/java-config-2/current-system-vm', 'CONFIG_PROTECT_MASK': '/etc/sandbox.d /etc/fonts/fonts.conf /etc/gentoo-release /etc/dconf /etc/ca-certificates.conf /etc/revdep-rebuild', 'TERMCAP': 'SC|screen.xterm-256color|VT 100/ANSI X3.64 virtual terminal:DO=\\E[%dB:LE=\\E[%dD:RI=\\E[%dC:UP=\\E[%dA:bs:bt=\\E[Z:cd=\\E[J:ce=\\E[K:cl=\\E[H\\E[J:cm=\\E[%i%d;%dH:ct=\\E[3g:do=^J:nd=\\E[C:pt:rc=\\E8:rs=\\Ec:sc=\\E7:st=\\EH:up=\\EM:le=^H:bl=^G:cr=^M:it#8:ho=\\E[H:nw=\\EE:ta=^I:is=\\E)0:li#64:co#238:am:xn:xv:LP:sr=\\EM:al=\\E[L:AL=\\E[%dL:cs=\\E[%i%d;%dr:dl=\\E[M:DL=\\E[%dM:dc=\\E[P:DC=\\E[%dP:im=\\E[4h:ei=\\E[4l:mi:IC=\\E[%d@:ks=\\E[?1h\\E=:ke=\\E[?1l\\E>:vi=\\E[?25l:ve=\\E[34h\\E[?25h:vs=\\E[34l:ti=\\E[?1049h:te=\\E[?1049l:us=\\E[4m:ue=\\E[24m:so=\\E[3m:se=\\E[23m:mb=\\E[5m:md=\\E[1m:mh=\\E[2m:mr=\\E[7m:me=\\E[m:ms:Co#8:pa#64:AF=\\E[3%dm:AB=\\E[4%dm:op=\\E[39;49m:AX:vb=\\Eg:G0:as=\\E(0:ae=\\E(B:ac=\\140\\140aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~..--++,,hhII00:po=\\E[5i:pf=\\E[4i:Z0=\\E[?3h:Z1=\\E[?3l:Km=\\E[<:k0=\\E[10~:k1=\\EOP:k2=\\EOQ:k3=\\EOR:k4=\\EOS:k5=\\E[15~:k6=\\E[17~:k7=\\E[18~:k8=\\E[19~:k9=\\E[20~:k;=\\E[21~:F1=\\E[23~:F2=\\E[24~:kB=\\E[Z:kh=\\E[1~:@1=\\E[1~:kH=\\E[4~:@7=\\E[4~:kN=\\E[6~:kP=\\E[5~:kI=\\E[2~:kD=\\E[3~:ku=\\EOA:kd=\\EOB:kr=\\EOC:kl=\\EOD:km:', 'JAVA_HOME': '/etc/java-config-2/current-system-vm', 'CSF_MIGRATION_TYPES': '/usr/share/opencascade/resources/StdResource/MigrationSheet.txt', 'WINDOW': '3', 'ANT_HOME': '/usr/share/ant', 'DESKTOP_SESSION': 'xfce', 'CSF_STEPDefaults': '/usr/share/opencascade/resources/XSTEPResource', 'EDITOR': 'vim', 'GTK_MODULES': 'canberra-gtk-module', 'XDG_SEAT': 'seat0', 'CONFIG_PROTECT': '/var/bind /usr/share/gnupg/qualified.txt /usr/share/easy-rsa /usr/share/config /usr/lib64/libreoffice/program/sofficerc', 'XDG_SESSION_DESKTOP': 'XFCE', 'XDG_SESSION_TYPE': 'x11', 'PANEL_GDK_CORE_DEVICE_EVENTS': '0', 'MANPATH': '/etc/java-config-2/current-system-vm/man:/usr/share/gcc-data/x86_64-pc-linux-gnu/13/man:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.41/man:/etc/java-config-2/current-system-vm/man/:/usr/local/share/man:/usr/share/man:/usr/lib/rust/man:/usr/lib/llvm/17/share/man:/etc/eselect/wine/share/man', 'XAUTHORITY': '/tmp/xauth_CEynPY', 'CSF_StandardLiteDefaults': '/usr/share/opencascade/resources/StdResource', 'OPENCL_PROFILE': 'nvidia', 'MOTD_SHOWN': 'pam', 'GDM_LANG': 'C.UTF-8', 'LANG': 'en_US.UTF-8', 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:su=37;41:sg=30;43:ca=00:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.avif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.cfg=00;32:*.conf=00;32:*.diff=00;32:*.doc=00;32:*.ini=00;32:*.log=00;32:*.patch=00;32:*.pdf=00;32:*.ps=00;32:*.tex=00;32:*.txt=00;32:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:*~=00;90:*#=00;90:*.bak=00;90:*.crdownload=00;90:*.dpkg-dist=00;90:*.dpkg-new=00;90:*.dpkg-old=00;90:*.dpkg-tmp=00;90:*.old=00;90:*.orig=00;90:*.part=00;90:*.rej=00;90:*.rpmnew=00;90:*.rpmorig=00;90:*.rpmsave=00;90:*.swp=00;90:*.tmp=00;90:*.ucf-dist=00;90:*.ucf-new=00;90:*.ucf-old=00;90:', 'XDG_CURRENT_DESKTOP': 'XFCE', 'VTE_VERSION': '7402', 'CSF_GraphicShr': '/usr/lib64/opencascade/libTKOpenGl.so', 'OPENGL_PROFILE': 'xorg-x11', 'XDG_SEAT_PATH': '/org/freedesktop/DisplayManager/Seat0', 'QT_GRAPHICSSYSTEM': 'raster', 'XZ_OPT': '-T0', 'CSF_ShadersDirectory': '/usr/share/opencascade/resources/Shaders', 'CSF_XmlOcafResource': '/usr/share/opencascade/resources/XmlOcafResource', 'INFOPATH': '/usr/share/gcc-data/x86_64-pc-linux-gnu/13/info:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.41/info:/usr/share/autoconf-2.72/info:/usr/share/autoconf-2.13/info:/usr/share/automake-1.16.5/info:/usr/share/info', 'MOZ_GMP_PATH': '/usr/lib64/nsbrowser/plugins/gmp-gmpopenh264/system-installed', 'CSF_SHMessage': '/usr/share/opencascade/resources/SHMessage', 'JAVAC': '/etc/java-config-2/current-system-vm/bin/javac', 'XDG_SESSION_CLASS': 'user', 'VBOX_SDK_PATH': '/usr/lib64/virtualbox/sdk', 'ANDROID_HOME': '/opt/android-sdk-update-manager', 'TERM': 'screen.xterm-256color', 'VBOX_INSTALL_PATH': '/usr/lib64/virtualbox', 'LESSOPEN': '|lesspipe %s', 'ANDROID_SWT': '/usr/share/swt-3.7/lib', 'CSF_StandardDefaults': '/usr/share/opencascade/resources/StdResource', 'CSF_IGESDefaults': '/usr/share/opencascade/resources/XSTEPResource', 'MANPAGER': 'manpager', 'NPM_CONFIG_GLOBALCONFIG': '/etc/npm/npmrc', 'DISPLAY': ':0.0', 'CSF_XCAFDefaults': '/usr/share/opencascade/resources/StdResource', 'SHLVL': '2', 'PAGER': '/usr/bin/less', 'LEX': 'flex', 'XDG_VTNR': '2', 'CSF_PluginDefaults': '/usr/share/opencascade/resources/StdResource', 'XDG_SESSION_ID': '2', 'CASROOT': '/usr', 'XDG_RUNTIME_DIR': '/run/user/1026', 'GCC_SPECS': '', 'GSETTINGS_BACKEND': 'dconf', 'PS1': '\\[\\033[01;32m\\]OE $MACHINE@$DISTRO\\[\\033[01;34m\\] \\w \\$\\[\\033[00m\\] ', 'CSF_XSMessage': '/usr/share/opencascade/resources/XSMessage', 'XDG_DATA_DIRS': '/usr/local/share:/usr/share:/etc/eselect/wine/share:/var/lib/snapd/desktop', 'CSF_UnitsDefinition': '/usr/share/opencascade/resources/UnitsAPI/Units.dat', 'STY': '2067.x', 'VBOX_APP_HOME': '/usr/lib64/virtualbox', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1026/bus', 'INITPWD': '/OE/build/luneos-kirkstone', 'MAIL': '/var/mail/martin', 'OLDPWD': '/home/martin-gentoo', '_': '/OE/build/luneos-kirkstone/bitbake/bin/bitbake'}, ['/OE/build/luneos-kirkstone/bitbake/bin/bitbake', '-k', 'zlib-native']] /OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
if isinstance(node.args[0], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229: DeprecationWarning: Attribute s is deprecated and will be removed in Python 3.14; use value instead
varname = node.args[0].s
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:230: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:233: DeprecationWarning: Attribute s is deprecated and will be removed in Python 3.14; use value instead
self.contains[varname].add(node.args[1].s)
/usr/lib/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning: This process (pid=4184584) is multi-threaded, use of fork() may lead to deadlocks in the child.
self.pid = os.fork()
4184584 16:47:45.108449 Command Completed
4184584 16:47:45.112267 Running command ['getVariable', 'BBINCLUDELOGS']
4184584 16:47:45.113080 Command Completed
4184584 16:47:45.113399 Running command ['getVariable', 'BBINCLUDELOGS_LINES']
4184584 16:47:45.113925 Command Completed
4184584 16:47:45.114218 Running command ['getSetVariable', 'BB_CONSOLELOG']
4184584 16:47:45.114956 Command Completed
4184584 16:47:45.115441 Running command ['getSetVariable', 'BB_LOGCONFIG']
4184584 16:47:45.116590 Command Completed
4184584 16:47:45.122232 Running command ['getUIHandlerNum']
4184584 16:47:45.122805 Command Completed
4184584 16:47:45.123194 Running command ['setEventMask', 1, 20, {'BitBake.SigGen.HashEquiv': 19, 'BitBake.RunQueue.HashEquiv': 19}, ['bb.runqueue.runQueueExitWait', 'bb.event.LogExecTTY', 'logging.LogRecord', 'bb.build.TaskFailed', 'bb.build.TaskBase', 'bb.event.ParseStarted', 'bb.event.ParseProgress', 'bb.event.ParseCompleted', 'bb.event.CacheLoadStarted', 'bb.event.CacheLoadProgress', 'bb.event.CacheLoadCompleted', 'bb.command.CommandFailed', 'bb.command.CommandExit', 'bb.command.CommandCompleted', 'bb.cooker.CookerExit', 'bb.event.MultipleProviders', 'bb.event.NoProvider', 'bb.runqueue.sceneQueueTaskStarted', 'bb.runqueue.runQueueTaskStarted', 'bb.runqueue.runQueueTaskFailed', 'bb.runqueue.sceneQueueTaskFailed', 'bb.event.BuildBase', 'bb.build.TaskStarted', 'bb.build.TaskSucceeded', 'bb.build.TaskFailedSilent', 'bb.build.TaskProgress', 'bb.event.ProcessStarted', 'bb.event.ProcessProgress', 'bb.event.ProcessFinished']]
4184584 16:47:45.123558 Command Completed
4184584 16:47:45.123863 Running command ['getVariable', 'BB_DEFAULT_TASK']
4184584 16:47:45.124228 Command Completed
4184584 16:47:45.124528 Running command ['setConfig', 'cmd', 'build']
4184584 16:47:45.124858 Command Completed
4184584 16:47:45.125140 Running command ['buildTargets', ['zlib-native'], 'build']
4184584 16:47:45.125294 Command Completed
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
if isinstance(node.args[0], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229: DeprecationWarning: Attribute s is deprecated and will be removed in Python 3.14; use value instead
varname = node.args[0].s
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:230: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:233: DeprecationWarning: Attribute s is deprecated and will be removed in Python 3.14; use value instead
self.contains[varname].add(node.args[1].s)
/usr/lib/python3.12/asyncio/streams.py:413: ResourceWarning: unclosed <StreamWriter transport=<_SelectorSocketTransport closing fd=18 read=idle write=<idle, bufsize=0>> reader=<StreamReader transport=<_SelectorSocketTransport closing fd=18 read=idle write=<idle, bufsize=0>>>>
warnings.warn(f"unclosed {self!r}", ResourceWarning)
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
if isinstance(node.args[0], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229: DeprecationWarning: Attribute s is deprecated and will be removed in Python 3.14; use value instead
varname = node.args[0].s
4184584 16:48:37.909973 Processing Client
4184584 16:48:37.910125 Disconnecting Client
4184584 16:48:37.910729 No timeout, exiting.
4184584 16:48:38.011341 Exiting

Quoted 1 line

It is reproducible with nanbield as well and fixed in scarthgap, will bisect it when time allows.

Quoted 6 lines

bisect was simpler than expected as nanbield is still compatible with newer bitbake and there weren't that many changes.

nanbield as well as kirkstone can be fixed by cherry-picking:
https://git.openembedded.org/bitbake/commit/?id=d01d684a0 asyncrpc: Add context manager API

+ Joshua if he sees some risks in cherry-picking this to stable releases.

While we wait for Joshua to comment I'll do a little autobuilder testing to see if anything blows up with this patch.

Steve

#19 lore raw
In reply to Martin Jansa
Quoted 231 lines · Martin Jansa
Quoted 224 lines · Martin Jansa via lists.openembedded.org

FWIW: I see the processes running after bitbake exit only in kirkstone and only with
PRSERV_HOST = "localhost:0"
if I drop PRSERV_HOST then all 3 bitbake-server processes exist together with bitbake itself.

luneos-kirkstone $ rm -f bitbake-cookerdaemon.log
luneos-kirkstone $ ps aux | grep luneos-kirkstone
martin 4184043 0.0 0.0 9124 2048 pts/4 S+ 16:47 0:00 grep
--colour=auto luneos-kirkstone
luneos-kirkstone $ bitbake -k zlib-native
NOTE: Started PRServer with DBfile:
/OE/build/luneos-kirkstone/cache/prserv.sqlite3, Address: 127.0.0.1:35811,
PID: 4184924
Loading cache: 100% |##########################################################################################################################################################################################################|
Time: 0:00:04
Loaded 4570 entries from dependency cache.
...
Sstate summary: Wanted 10 Local 10 Mirrors 0 Missed 0 Current 11 (100%
match, 100% complete)
NOTE: Executing Tasks
NOTE: Tasks Summary: Attempted 92 tasks of which 92 didn't need to be
rerun and all succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 15 seconds
luneos-kirkstone $ ps aux | grep luneos-kirkstone
martin 51324 0.0 0.0 9124 2048 pts/4 S+ 16:54 0:00 grep
--colour=auto luneos-kirkstone
martin 4184584 8.9 0.4 668196 536892 ? Sl 16:47 0:37
bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
/OE/build/luneos-kirkstone/bitbake.lock
/OE/build/luneos-kirkstone/bitbake.sock 0 None 0
martin 4184924 0.0 0.1 340280 196428 ? S 16:47 0:00
bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
/OE/build/luneos-kirkstone/bitbake.lock
/OE/build/luneos-kirkstone/bitbake.sock 0 None 0
martin 4184927 0.0 0.1 340328 194440 ? S 16:47 0:00
bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server
decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
/OE/build/luneos-kirkstone/bitbake.lock
/OE/build/luneos-kirkstone/bitbake.sock 0 None 0

luneos-kirkstone $ cat bitbake-cookerdaemon.log
4184584 16:47:38.367593 --- Starting bitbake server pid 4184584 at

2024-01-05 16:47:38.367526 ---

4184584 16:47:38.370315 Started bitbake server pid 4184584
4184584 16:47:38.370831 Entering server connection loop
4184584 16:47:38.372043 Accepting [<socket.socket fd=6, family=1, type=1,
proto=0, laddr=bitbake.sock>] ([])
4184584 16:47:38.376431 Processing Client
4184584 16:47:38.376633 Connecting Client
4184584 16:47:38.377632 Running command ['setFeatures', [2, 1]]
4184584 16:47:38.379355 Command Completed
4184584 16:47:38.379868 Running command ['updateConfig', {'halt': False,
'force': False, 'invalidate_stamp': None, 'dry_run': False,
'dump_signatures': [], 'extra_assume_provided': [], 'profile': False,
'prefile': [], 'postfile': [], 'server_timeout': None, 'nosetscene': False,
'setsceneonly': False, 'skipsetscene': False, 'runall': None, 'runonly':
None, 'writeeventlog': None, 'build_verbose_shell': False,
'build_verbose_stdout': False, 'default_loglevel': 20, 'debug_domains':
{}}, {'SHELL': '/bin/bash', 'BB_ENV_PASSTHROUGH_ADDITIONS': 'MACHINE DISTRO
TCMODE TCLIBC http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY no_proxy
SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS
PARALLEL_MAKE GIT_PROXY_COMMAND GIT_PROXY_IGNORE SOCKS5_PASSWD SOCKS5_USER
WEBOS_DISTRO_BUILD_ID PSEUDO_DISABLED PSEUDO_BUILD', 'DISTRO': 'luneos',
'SSH_AUTH_SOCK': '/tmp/ssh-XXXXXX2ZxYBo/agent.1893', 'SSH_AGENT_PID':
'1894', 'PWD': '/OE/build/luneos-kirkstone', 'LOGNAME': 'martin', 'HOME':
'/home/martin-gentoo', 'MACHINE': 'qemux86-64', 'USER': 'martin', 'PATH':
'/OE/build/luneos-kirkstone/openembedded-core/scripts:/OE/build/luneos-kirkstone/bitbake/bin:/home/martin-gentoo/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/17/bin:/opt/android-sdk-update-manager/tools:/opt/android-sdk-update-manager/platform-tools:/etc/eselect/wine/bin:/usr/lib64/opencascade/bin:/var/lib/snapd/snap/bin',
'LC_ALL': 'en_US.UTF-8', 'SESSION_MANAGER': 'local/jama:@/tmp/.ICE-unix/1838,unix/jama:/tmp/.ICE-unix/1838',
'WINDOWID': '35651587', 'COLORTERM': 'truecolor',
'CSF_MDTVTexturesDirectory': '/usr/share/opencascade/resources/Textures',
'MACHINES': 'qemux86-64', 'XDG_CONFIG_DIRS': '/etc/xdg', 'LESS': '-R -M
--shift 5', 'XDG_SESSION_PATH': '/org/freedesktop/DisplayManager/Session1',
'XDG_MENU_PREFIX': 'xfce-', 'JDK_HOME':
'/etc/java-config-2/current-system-vm', 'CONFIG_PROTECT_MASK':
'/etc/sandbox.d /etc/fonts/fonts.conf /etc/gentoo-release /etc/dconf
/etc/ca-certificates.conf /etc/revdep-rebuild', 'TERMCAP':
'SC|screen.xterm-256color|VT 100/ANSI X3.64 virtual
terminal:DO=\\E[%dB:LE=\\E[%dD:RI=\\E[%dC:UP=\\E[%dA:bs:bt=\\E[Z:cd=\\E[J:ce=\\E[K:cl=\\E[H\\E[J:cm=\\E[%i%d;%dH:ct=\\E[3g:do=^J:nd=\\E[C:pt:rc=\\E8:rs=\\Ec:sc=\\E7:st=\\EH:up=\\EM:le=^H:bl=^G:cr=^M:it#8:ho=\\E[H:nw=\\EE:ta=^I:is=\\E)0:li#64:co#238:am:xn:xv:LP:sr=\\EM:al=\\E[L:AL=\\E[%dL:cs=\\E[%i%d;%dr:dl=\\E[M:DL=\\E[%dM:dc=\\E[P:DC=\\E[%dP:im=\\E[4h:ei=\\E[4l:mi:IC=\\E[%d@:ks=\\E[?1h\\E=:ke=\\E[?1l\\E>:vi=\\E[?25l:ve=\\E[34h\\E[?25h:vs=\\E[34l:ti=\\E[?1049h:te=\\E[?1049l:us=\\E[4m:ue=\\E[24m:so=\\E[3m:se=\\E[23m:mb=\\E[5m:md=\\E[1m:mh=\\E[2m:mr=\\E[7m:me=\\E[m:ms:Co#8:pa#64:AF=\\E[3%dm:AB=\\E[4%dm:op=\\E[39;49m:AX:vb=\\Eg:G0:as=\\E(0:ae=\\E(B:ac=\\140\\140aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~..--++,,hhII00:po=\\E[5i:pf=\\E[4i:Z0=\\E[?3h:Z1=\\E[?3l:Km=\\E[<:k0=\\E[10~:k1=\\EOP:k2=\\EOQ:k3=\\EOR:k4=\\EOS:k5=\\E[15~:k6=\\E[17~:k7=\\E[18~:k8=\\E[19~:k9=\\E[20~:k;=\\E[21~:F1=\\E[23~:F2=\\E[24~:kB=\\E[Z:kh=\\E[1~:@1=\\E[1~:kH=\\E[4~:@7=\\E[4~:kN=\\E[6~:kP=\\E[5~:kI=\\E[2~:kD=\\E[3~:ku=\\EOA:kd=\\EOB:kr=\\EOC:kl=\\EOD:km:',
'JAVA_HOME': '/etc/java-config-2/current-system-vm', 'CSF_MIGRATION_TYPES':
'/usr/share/opencascade/resources/StdResource/MigrationSheet.txt',
'WINDOW': '3', 'ANT_HOME': '/usr/share/ant', 'DESKTOP_SESSION': 'xfce',
'CSF_STEPDefaults': '/usr/share/opencascade/resources/XSTEPResource',
'EDITOR': 'vim', 'GTK_MODULES': 'canberra-gtk-module', 'XDG_SEAT': 'seat0',
'CONFIG_PROTECT': '/var/bind /usr/share/gnupg/qualified.txt
/usr/share/easy-rsa /usr/share/config
/usr/lib64/libreoffice/program/sofficerc', 'XDG_SESSION_DESKTOP': 'XFCE',
'XDG_SESSION_TYPE': 'x11', 'PANEL_GDK_CORE_DEVICE_EVENTS': '0', 'MANPATH':
'/etc/java-config-2/current-system-vm/man:/usr/share/gcc-data/x86_64-pc-linux-gnu/13/man:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.41/man:/etc/java-config-2/current-system-vm/man/:/usr/local/share/man:/usr/share/man:/usr/lib/rust/man:/usr/lib/llvm/17/share/man:/etc/eselect/wine/share/man',
'XAUTHORITY': '/tmp/xauth_CEynPY', 'CSF_StandardLiteDefaults':
'/usr/share/opencascade/resources/StdResource', 'OPENCL_PROFILE': 'nvidia',
'MOTD_SHOWN': 'pam', 'GDM_LANG': 'C.UTF-8', 'LANG': 'en_US.UTF-8',
'LS_COLORS':
'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:su=37;41:sg=30;43:ca=00:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.avif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.cfg=00;32:*.conf=00;32:*.diff=00;32:*.doc=00;32:*.ini=00;32:*.log=00;32:*.patch=00;32:*.pdf=00;32:*.ps=00;32:*.tex=00;32:*.txt=00;32:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:*~=00;90:*#=00;90:*.bak=00;90:*.crdownload=00;90:*.dpkg-dist=00;90:*.dpkg-new=00;90:*.dpkg-old=00;90:*.dpkg-tmp=00;90:*.old=00;90:*.orig=00;90:*.part=00;90:*.rej=00;90:*.rpmnew=00;90:*.rpmorig=00;90:*.rpmsave=00;90:*.swp=00;90:*.tmp=00;90:*.ucf-dist=00;90:*.ucf-new=00;90:*.ucf-old=00;90:', 'XDG_CURRENT_DESKTOP': 'XFCE', 'VTE_VERSION': '7402', 'CSF_GraphicShr':
'/usr/lib64/opencascade/libTKOpenGl.so', 'OPENGL_PROFILE': 'xorg-x11',
'XDG_SEAT_PATH': '/org/freedesktop/DisplayManager/Seat0',
'QT_GRAPHICSSYSTEM': 'raster', 'XZ_OPT': '-T0', 'CSF_ShadersDirectory':
'/usr/share/opencascade/resources/Shaders', 'CSF_XmlOcafResource':
'/usr/share/opencascade/resources/XmlOcafResource', 'INFOPATH':
'/usr/share/gcc-data/x86_64-pc-linux-gnu/13/info:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.41/info:/usr/share/autoconf-2.72/info:/usr/share/autoconf-2.13/info:/usr/share/automake-1.16.5/info:/usr/share/info',
'MOZ_GMP_PATH':
'/usr/lib64/nsbrowser/plugins/gmp-gmpopenh264/system-installed',
'CSF_SHMessage': '/usr/share/opencascade/resources/SHMessage', 'JAVAC':
'/etc/java-config-2/current-system-vm/bin/javac', 'XDG_SESSION_CLASS':
'user', 'VBOX_SDK_PATH': '/usr/lib64/virtualbox/sdk', 'ANDROID_HOME':
'/opt/android-sdk-update-manager', 'TERM': 'screen.xterm-256color',
'VBOX_INSTALL_PATH': '/usr/lib64/virtualbox', 'LESSOPEN': '|lesspipe %s',
'ANDROID_SWT': '/usr/share/swt-3.7/lib', 'CSF_StandardDefaults':
'/usr/share/opencascade/resources/StdResource', 'CSF_IGESDefaults':
'/usr/share/opencascade/resources/XSTEPResource', 'MANPAGER': 'manpager',
'NPM_CONFIG_GLOBALCONFIG': '/etc/npm/npmrc', 'DISPLAY': ':0.0',
'CSF_XCAFDefaults': '/usr/share/opencascade/resources/StdResource',
'SHLVL': '2', 'PAGER': '/usr/bin/less', 'LEX': 'flex', 'XDG_VTNR': '2',
'CSF_PluginDefaults': '/usr/share/opencascade/resources/StdResource',
'XDG_SESSION_ID': '2', 'CASROOT': '/usr', 'XDG_RUNTIME_DIR':
'/run/user/1026', 'GCC_SPECS': '', 'GSETTINGS_BACKEND': 'dconf', 'PS1':
'\\[\\033[01;32m\\]OE $MACHINE@$DISTRO\\[\\033[01;34m\\] \\w
\\$\\[\\033[00m\\] ', 'CSF_XSMessage':
'/usr/share/opencascade/resources/XSMessage', 'XDG_DATA_DIRS':
'/usr/local/share:/usr/share:/etc/eselect/wine/share:/var/lib/snapd/desktop',
'CSF_UnitsDefinition':
'/usr/share/opencascade/resources/UnitsAPI/Units.dat', 'STY': '2067.x',
'VBOX_APP_HOME': '/usr/lib64/virtualbox', 'DBUS_SESSION_BUS_ADDRESS':
'unix:path=/run/user/1026/bus', 'INITPWD': '/OE/build/luneos-kirkstone',
'MAIL': '/var/mail/martin', 'OLDPWD': '/home/martin-gentoo', '_':
'/OE/build/luneos-kirkstone/bitbake/bin/bitbake'},
['/OE/build/luneos-kirkstone/bitbake/bin/bitbake', '-k', 'zlib-native']]
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228:
DeprecationWarning: ast.Str is deprecated and will be removed in Python
3.14; use ast.Constant instead
if isinstance(node.args[0], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229:
DeprecationWarning: Attribute s is deprecated and will be removed in Python
3.14; use value instead
varname = node.args[0].s
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:230:
DeprecationWarning: ast.Str is deprecated and will be removed in Python
3.14; use ast.Constant instead
if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:233:
DeprecationWarning: Attribute s is deprecated and will be removed in Python
3.14; use value instead
self.contains[varname].add(node.args[1].s)
/usr/lib/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning:
This process (pid=4184584) is multi-threaded, use of fork() may lead to
deadlocks in the child.
self.pid = os.fork()
4184584 16:47:45.108449 Command Completed
4184584 16:47:45.112267 Running command ['getVariable', 'BBINCLUDELOGS']
4184584 16:47:45.113080 Command Completed
4184584 16:47:45.113399 Running command ['getVariable',
'BBINCLUDELOGS_LINES']
4184584 16:47:45.113925 Command Completed
4184584 16:47:45.114218 Running command ['getSetVariable',
'BB_CONSOLELOG']
4184584 16:47:45.114956 Command Completed
4184584 16:47:45.115441 Running command ['getSetVariable', 'BB_LOGCONFIG']
4184584 16:47:45.116590 Command Completed
4184584 16:47:45.122232 Running command ['getUIHandlerNum']
4184584 16:47:45.122805 Command Completed
4184584 16:47:45.123194 Running command ['setEventMask', 1, 20,
{'BitBake.SigGen.HashEquiv': 19, 'BitBake.RunQueue.HashEquiv': 19},
['bb.runqueue.runQueueExitWait', 'bb.event.LogExecTTY',
'logging.LogRecord', 'bb.build.TaskFailed', 'bb.build.TaskBase',
'bb.event.ParseStarted', 'bb.event.ParseProgress',
'bb.event.ParseCompleted', 'bb.event.CacheLoadStarted',
'bb.event.CacheLoadProgress', 'bb.event.CacheLoadCompleted',
'bb.command.CommandFailed', 'bb.command.CommandExit',
'bb.command.CommandCompleted', 'bb.cooker.CookerExit',
'bb.event.MultipleProviders', 'bb.event.NoProvider',
'bb.runqueue.sceneQueueTaskStarted', 'bb.runqueue.runQueueTaskStarted',
'bb.runqueue.runQueueTaskFailed', 'bb.runqueue.sceneQueueTaskFailed',
'bb.event.BuildBase', 'bb.build.TaskStarted', 'bb.build.TaskSucceeded',
'bb.build.TaskFailedSilent', 'bb.build.TaskProgress',
'bb.event.ProcessStarted', 'bb.event.ProcessProgress',
'bb.event.ProcessFinished']]
4184584 16:47:45.123558 Command Completed
4184584 16:47:45.123863 Running command ['getVariable', 'BB_DEFAULT_TASK']
4184584 16:47:45.124228 Command Completed
4184584 16:47:45.124528 Running command ['setConfig', 'cmd', 'build']
4184584 16:47:45.124858 Command Completed
4184584 16:47:45.125140 Running command ['buildTargets', ['zlib-native'],
'build']
4184584 16:47:45.125294 Command Completed
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228:
DeprecationWarning: ast.Str is deprecated and will be removed in Python
3.14; use ast.Constant instead
if isinstance(node.args[0], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229:
DeprecationWarning: Attribute s is deprecated and will be removed in Python
3.14; use value instead
varname = node.args[0].s
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:230:
DeprecationWarning: ast.Str is deprecated and will be removed in Python
3.14; use ast.Constant instead
if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:233:
DeprecationWarning: Attribute s is deprecated and will be removed in Python
3.14; use value instead
self.contains[varname].add(node.args[1].s)
/usr/lib/python3.12/asyncio/streams.py:413: ResourceWarning: unclosed
<StreamWriter transport=<_SelectorSocketTransport closing fd=18 read=idle
write=<idle, bufsize=0>> reader=<StreamReader
transport=<_SelectorSocketTransport closing fd=18 read=idle write=<idle,
bufsize=0>>>>
warnings.warn(f"unclosed {self!r}", ResourceWarning)
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228:
DeprecationWarning: ast.Str is deprecated and will be removed in Python
3.14; use ast.Constant instead
if isinstance(node.args[0], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229:
DeprecationWarning: Attribute s is deprecated and will be removed in Python
3.14; use value instead
varname = node.args[0].s
4184584 16:48:37.909973 Processing Client
4184584 16:48:37.910125 Disconnecting Client
4184584 16:48:37.910729 No timeout, exiting.
4184584 16:48:38.011341 Exiting

Quoted 3 lines

It is reproducible with nanbield as well and fixed in scarthgap, will bisect it when time allows.

Quoted 9 lines

bisect was simpler than expected as nanbield is still compatible with newer bitbake and there weren't that many changes.

nanbield as well as kirkstone can be fixed by cherry-picking:
https://git.openembedded.org/bitbake/commit/?id=d01d684a0 asyncrpc: Add context manager API

+ Joshua if he sees some risks in cherry-picking this to stable releases.

It looks ok. Must be the prserv client that's hanging then?

Quoted 4 lines

Cheers

A message that is not in the archive (sent off-list or not yet received): [email protected]

#21 lore raw
Quoted 91 lines · Steve Sakoman via lists.openembedded.org
Quoted 89 lines · Martin Jansa
Quoted 82 lines · Martin Jansa via lists.openembedded.org

FWIW: I see the processes running after bitbake exit only in kirkstone and only with PRSERV_HOST = "localhost:0"
if I drop PRSERV_HOST then all 3 bitbake-server processes exist together with bitbake itself.

luneos-kirkstone $ rm -f bitbake-cookerdaemon.log
luneos-kirkstone $ ps aux | grep luneos-kirkstone
martin 4184043 0.0 0.0 9124 2048 pts/4 S+ 16:47 0:00 grep --colour=auto luneos-kirkstone
luneos-kirkstone $ bitbake -k zlib-native
NOTE: Started PRServer with DBfile: /OE/build/luneos-kirkstone/cache/prserv.sqlite3, Address: 127.0.0.1:35811, PID: 4184924
Loading cache: 100% |##########################################################################################################################################################################################################| Time: 0:00:04 Loaded 4570 entries from dependency cache.
...
Sstate summary: Wanted 10 Local 10 Mirrors 0 Missed 0 Current 11 (100% match, 100% complete)
NOTE: Executing Tasks
NOTE: Tasks Summary: Attempted 92 tasks of which 92 didn't need to be rerun and all succeeded.
NOTE: Writing buildhistory
NOTE: Writing buildhistory took: 15 seconds
luneos-kirkstone $ ps aux | grep luneos-kirkstone

martin     51324  0.0  0.0   9124  2048 pts/4    S+   16:54   0:00 grep --colour=auto luneos-kirkstone
martin   4184584  8.9  0.4 668196 536892 ?       Sl   16:47   0:37 bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log /OE/build/luneos-kirkstone/bitbake.lock /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
martin   4184924  0.0  0.1 340280 196428 ?       S    16:47   0:00 bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log /OE/build/luneos-kirkstone/bitbake.lock /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
martin   4184927  0.0  0.1 340328 194440 ?       S    16:47   0:00 bitbake-server /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server decafbad 3 5 /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log /OE/build/luneos-kirkstone/bitbake.lock /OE/build/luneos-kirkstone/bitbake.sock 0 None 0

luneos-kirkstone $ cat bitbake-cookerdaemon.log
4184584 16:47:38.367593 --- Starting bitbake server pid 4184584 at 2024-01-05 16:47:38.367526 ---
4184584 16:47:38.370315 Started bitbake server pid 4184584
4184584 16:47:38.370831 Entering server connection loop
4184584 16:47:38.372043 Accepting [<socket.socket fd=6, family=1, type=1, proto=0, laddr=bitbake.sock>] ([])
4184584 16:47:38.376431 Processing Client
4184584 16:47:38.376633 Connecting Client
4184584 16:47:38.377632 Running command ['setFeatures', [2, 1]]
4184584 16:47:38.379355 Command Completed
4184584 16:47:38.379868 Running command ['updateConfig', {'halt': False, 'force': False, 'invalidate_stamp': None, 'dry_run': False, 'dump_signatures': [], 'extra_assume_provided': [], 'profile': False, 'prefile': [], 'postfile': [], 'server_timeout': None, 'nosetscene': False, 'setsceneonly': False, 'skipsetscene': False, 'runall': None, 'runonly': None, 'writeeventlog': None, 'build_verbose_shell': False, 'build_verbose_stdout': False, 'default_loglevel': 20, 'debug_domains': {}}, {'SHELL': '/bin/bash', 'BB_ENV_PASSTHROUGH_ADDITIONS': 'MACHINE DISTRO TCMODE TCLIBC http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS PARALLEL_MAKE GIT_PROXY_COMMAND GIT_PROXY_IGNORE SOCKS5_PASSWD SOCKS5_USER WEBOS_DISTRO_BUILD_ID PSEUDO_DISABLED PSEUDO_BUILD', 'DISTRO': 'luneos', 'SSH_AUTH_SOCK': '/tmp/ssh-XXXXXX2ZxYBo/agent.1893', 'SSH_AGENT_PID': '1894', 'PWD': '/OE/build/luneos-kirkstone', 'LOGNAME': 'martin', 'HOME': '/home/martin-gentoo', 'MACHINE': 'qemux86-64', 'USER': 'martin', 'PATH': '/OE/build/luneos-kirkstone/openembedded-core/scripts:/OE/build/luneos-kirkstone/bitbake/bin:/home/martin-gentoo/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/17/bin:/opt/android-sdk-update-manager/tools:/opt/android-sdk-update-manager/platform-tools:/etc/eselect/wine/bin:/usr/lib64/opencascade/bin:/var/lib/snapd/snap/bin', 'LC_ALL': 'en_US.UTF-8', 'SESSION_MANAGER': 'local/jama:@/tmp/.ICE-unix/1838,unix/jama:/tmp/.ICE-unix/1838', 'WINDOWID': '35651587', 'COLORTERM': 'truecolor', 'CSF_MDTVTexturesDirectory': '/usr/share/opencascade/resources/Textures', 'MACHINES': 'qemux86-64', 'XDG_CONFIG_DIRS': '/etc/xdg', 'LESS': '-R -M --shift 5', 'XDG_SESSION_PATH': '/org/freedesktop/DisplayManager/Session1', 'XDG_MENU_PREFIX': 'xfce-', 'JDK_HOME': '/etc/java-config-2/current-system-vm', 'CONFIG_PROTECT_MASK': '/etc/sandbox.d /etc/fonts/fonts.conf /etc/gentoo-release /etc/dconf /etc/ca-certificates.conf /etc/revdep-rebuild', 'TERMCAP': 'SC|screen.xterm-256color|VT 100/ANSI X3.64 virtual terminal:DO=\\E[%dB:LE=\\E[%dD:RI=\\E[%dC:UP=\\E[%dA:bs:bt=\\E[Z:cd=\\E[J:ce=\\E[K:cl=\\E[H\\E[J:cm=\\E[%i%d;%dH:ct=\\E[3g:do=^J:nd=\\E[C:pt:rc=\\E8:rs=\\Ec:sc=\\E7:st=\\EH:up=\\EM:le=^H:bl=^G:cr=^M:it#8:ho=\\E[H:nw=\\EE:ta=^I:is=\\E)0:li#64:co#238:am:xn:xv:LP:sr=\\EM:al=\\E[L:AL=\\E[%dL:cs=\\E[%i%d;%dr:dl=\\E[M:DL=\\E[%dM:dc=\\E[P:DC=\\E[%dP:im=\\E[4h:ei=\\E[4l:mi:IC=\\E[%d@:ks=\\E[?1h\\E=:ke=\\E[?1l\\E>:vi=\\E[?25l:ve=\\E[34h\\E[?25h:vs=\\E[34l:ti=\\E[?1049h:te=\\E[?1049l:us=\\E[4m:ue=\\E[24m:so=\\E[3m:se=\\E[23m:mb=\\E[5m:md=\\E[1m:mh=\\E[2m:mr=\\E[7m:me=\\E[m:ms:Co#8:pa#64:AF=\\E[3%dm:AB=\\E[4%dm:op=\\E[39;49m:AX:vb=\\Eg:G0:as=\\E(0:ae=\\E(B:ac=\\140\\140aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~..--++,,hhII00:po=\\E[5i:pf=\\E[4i:Z0=\\E[?3h:Z1=\\E[?3l:Km=\\E[<:k0=\\E[10~:k1=\\EOP:k2=\\EOQ:k3=\\EOR:k4=\\EOS:k5=\\E[15~:k6=\\E[17~:k7=\\E[18~:k8=\\E[19~:k9=\\E[20~:k;=\\E[21~:F1=\\E[23~:F2=\\E[24~:kB=\\E[Z:kh=\\E[1~:@1=\\E[1~:kH=\\E[4~:@7=\\E[4~:kN=\\E[6~:kP=\\E[5~:kI=\\E[2~:kD=\\E[3~:ku=\\EOA:kd=\\EOB:kr=\\EOC:kl=\\EOD:km:', 'JAVA_HOME': '/etc/java-config-2/current-system-vm', 'CSF_MIGRATION_TYPES': '/usr/share/opencascade/resources/StdResource/MigrationSheet.txt', 'WINDOW': '3', 'ANT_HOME': '/usr/share/ant', 'DESKTOP_SESSION': 'xfce', 'CSF_STEPDefaults': '/usr/share/opencascade/resources/XSTEPResource', 'EDITOR': 'vim', 'GTK_MODULES': 'canberra-gtk-module', 'XDG_SEAT': 'seat0', 'CONFIG_PROTECT': '/var/bind /usr/share/gnupg/qualified.txt /usr/share/easy-rsa /usr/share/config /usr/lib64/libreoffice/program/sofficerc', 'XDG_SESSION_DESKTOP': 'XFCE', 'XDG_SESSION_TYPE': 'x11', 'PANEL_GDK_CORE_DEVICE_EVENTS': '0', 'MANPATH': '/etc/java-config-2/current-system-vm/man:/usr/share/gcc-data/x86_64-pc-linux-gnu/13/man:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.41/man:/etc/java-config-2/current-system-vm/man/:/usr/local/share/man:/usr/share/man:/usr/lib/rust/man:/usr/lib/llvm/17/share/man:/etc/eselect/wine/share/man', 'XAUTHORITY': '/tmp/xauth_CEynPY', 'CSF_StandardLiteDefaults': '/usr/share/opencascade/resources/StdResource', 'OPENCL_PROFILE': 'nvidia', 'MOTD_SHOWN': 'pam', 'GDM_LANG': 'C.UTF-8', 'LANG': 'en_US.UTF-8', 'LS_COLORS': 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:su=37;41:sg=30;43:ca=00:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.avif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.cfg=00;32:*.conf=00;32:*.diff=00;32:*.doc=00;32:*.ini=00;32:*.log=00;32:*.patch=00;32:*.pdf=00;32:*.ps=00;32:*.tex=00;32:*.txt=00;32:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:*~=00;90:*#=00;90:*.bak=00;90:*.crdownload=00;90:*.dpkg-dist=00;90:*.dpkg-new=00;90:*.dpkg-old=00;90:*.dpkg-tmp=00;90:*.old=00;90:*.orig=00;90:*.part=00;90:*.rej=00;90:*.rpmnew=00;90:*.rpmorig=00;90:*.rpmsave=00;90:*.swp=00;90:*.tmp=00;90:*.ucf-dist=00;90:*.ucf-new=00;90:*.ucf-old=00;90:', 'XDG_CURRENT_DESKTOP': 'XFCE', 'VTE_VERSION': '7402', 'CSF_GraphicShr': '/usr/lib64/opencascade/libTKOpenGl.so', 'OPENGL_PROFILE': 'xorg-x11', 'XDG_SEAT_PATH': '/org/freedesktop/DisplayManager/Seat0', 'QT_GRAPHICSSYSTEM': 'raster', 'XZ_OPT': '-T0', 'CSF_ShadersDirectory': '/usr/share/opencascade/resources/Shaders', 'CSF_XmlOcafResource': '/usr/share/opencascade/resources/XmlOcafResource', 'INFOPATH': '/usr/share/gcc-data/x86_64-pc-linux-gnu/13/info:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.41/info:/usr/share/autoconf-2.72/info:/usr/share/autoconf-2.13/info:/usr/share/automake-1.16.5/info:/usr/share/info', 'MOZ_GMP_PATH': '/usr/lib64/nsbrowser/plugins/gmp-gmpopenh264/system-installed', 'CSF_SHMessage': '/usr/share/opencascade/resources/SHMessage', 'JAVAC': '/etc/java-config-2/current-system-vm/bin/javac', 'XDG_SESSION_CLASS': 'user', 'VBOX_SDK_PATH': '/usr/lib64/virtualbox/sdk', 'ANDROID_HOME': '/opt/android-sdk-update-manager', 'TERM': 'screen.xterm-256color', 'VBOX_INSTALL_PATH': '/usr/lib64/virtualbox', 'LESSOPEN': '|lesspipe %s', 'ANDROID_SWT': '/usr/share/swt-3.7/lib', 'CSF_StandardDefaults': '/usr/share/opencascade/resources/StdResource', 'CSF_IGESDefaults': '/usr/share/opencascade/resources/XSTEPResource', 'MANPAGER': 'manpager', 'NPM_CONFIG_GLOBALCONFIG': '/etc/npm/npmrc', 'DISPLAY': ':0.0', 'CSF_XCAFDefaults': '/usr/share/opencascade/resources/StdResource', 'SHLVL': '2', 'PAGER': '/usr/bin/less', 'LEX': 'flex', 'XDG_VTNR': '2', 'CSF_PluginDefaults': '/usr/share/opencascade/resources/StdResource', 'XDG_SESSION_ID': '2', 'CASROOT': '/usr', 'XDG_RUNTIME_DIR': '/run/user/1026', 'GCC_SPECS': '', 'GSETTINGS_BACKEND': 'dconf', 'PS1': '\\[\\033[01;32m\\]OE $MACHINE@$DISTRO\\[\\033[01;34m\\] \\w \\$\\[\\033[00m\\] ', 'CSF_XSMessage': '/usr/share/opencascade/resources/XSMessage', 'XDG_DATA_DIRS': '/usr/local/share:/usr/share:/etc/eselect/wine/share:/var/lib/snapd/desktop', 'CSF_UnitsDefinition': '/usr/share/opencascade/resources/UnitsAPI/Units.dat', 'STY': '2067.x', 'VBOX_APP_HOME': '/usr/lib64/virtualbox', 'DBUS_SESSION_BUS_ADDRESS': 'unix:path=/run/user/1026/bus', 'INITPWD': '/OE/build/luneos-kirkstone', 'MAIL': '/var/mail/martin', 'OLDPWD': '/home/martin-gentoo', '_': '/OE/build/luneos-kirkstone/bitbake/bin/bitbake'}, ['/OE/build/luneos-kirkstone/bitbake/bin/bitbake', '-k', 'zlib-native']] /OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
if isinstance(node.args[0], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229: DeprecationWarning: Attribute s is deprecated and will be removed in Python 3.14; use value instead
varname = node.args[0].s
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:230: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:233: DeprecationWarning: Attribute s is deprecated and will be removed in Python 3.14; use value instead
self.contains[varname].add(node.args[1].s)
/usr/lib/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning: This process (pid=4184584) is multi-threaded, use of fork() may lead to deadlocks in the child.
self.pid = os.fork()
4184584 16:47:45.108449 Command Completed
4184584 16:47:45.112267 Running command ['getVariable', 'BBINCLUDELOGS']
4184584 16:47:45.113080 Command Completed
4184584 16:47:45.113399 Running command ['getVariable', 'BBINCLUDELOGS_LINES']
4184584 16:47:45.113925 Command Completed
4184584 16:47:45.114218 Running command ['getSetVariable', 'BB_CONSOLELOG']
4184584 16:47:45.114956 Command Completed
4184584 16:47:45.115441 Running command ['getSetVariable', 'BB_LOGCONFIG']
4184584 16:47:45.116590 Command Completed
4184584 16:47:45.122232 Running command ['getUIHandlerNum']
4184584 16:47:45.122805 Command Completed
4184584 16:47:45.123194 Running command ['setEventMask', 1, 20, {'BitBake.SigGen.HashEquiv': 19, 'BitBake.RunQueue.HashEquiv': 19}, ['bb.runqueue.runQueueExitWait', 'bb.event.LogExecTTY', 'logging.LogRecord', 'bb.build.TaskFailed', 'bb.build.TaskBase', 'bb.event.ParseStarted', 'bb.event.ParseProgress', 'bb.event.ParseCompleted', 'bb.event.CacheLoadStarted', 'bb.event.CacheLoadProgress', 'bb.event.CacheLoadCompleted', 'bb.command.CommandFailed', 'bb.command.CommandExit', 'bb.command.CommandCompleted', 'bb.cooker.CookerExit', 'bb.event.MultipleProviders', 'bb.event.NoProvider', 'bb.runqueue.sceneQueueTaskStarted', 'bb.runqueue.runQueueTaskStarted', 'bb.runqueue.runQueueTaskFailed', 'bb.runqueue.sceneQueueTaskFailed', 'bb.event.BuildBase', 'bb.build.TaskStarted', 'bb.build.TaskSucceeded', 'bb.build.TaskFailedSilent', 'bb.build.TaskProgress', 'bb.event.ProcessStarted', 'bb.event.ProcessProgress', 'bb.event.ProcessFinished']]
4184584 16:47:45.123558 Command Completed
4184584 16:47:45.123863 Running command ['getVariable', 'BB_DEFAULT_TASK']
4184584 16:47:45.124228 Command Completed
4184584 16:47:45.124528 Running command ['setConfig', 'cmd', 'build']
4184584 16:47:45.124858 Command Completed
4184584 16:47:45.125140 Running command ['buildTargets', ['zlib-native'], 'build']
4184584 16:47:45.125294 Command Completed
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
if isinstance(node.args[0], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229: DeprecationWarning: Attribute s is deprecated and will be removed in Python 3.14; use value instead
varname = node.args[0].s
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:230: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:233: DeprecationWarning: Attribute s is deprecated and will be removed in Python 3.14; use value instead
self.contains[varname].add(node.args[1].s)
/usr/lib/python3.12/asyncio/streams.py:413: ResourceWarning: unclosed <StreamWriter transport=<_SelectorSocketTransport closing fd=18 read=idle write=<idle, bufsize=0>> reader=<StreamReader transport=<_SelectorSocketTransport closing fd=18 read=idle write=<idle, bufsize=0>>>>
warnings.warn(f"unclosed {self!r}", ResourceWarning)
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228: DeprecationWarning: ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
if isinstance(node.args[0], ast.Str):
/OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229: DeprecationWarning: Attribute s is deprecated and will be removed in Python 3.14; use value instead
varname = node.args[0].s
4184584 16:48:37.909973 Processing Client
4184584 16:48:37.910125 Disconnecting Client
4184584 16:48:37.910729 No timeout, exiting.
4184584 16:48:38.011341 Exiting

Quoted 1 line

It is reproducible with nanbield as well and fixed in scarthgap, will bisect it when time allows.

bisect was simpler than expected as nanbield is still compatible with newer bitbake and there weren't that many changes.

Quoted 7 lines
Quoted 4 lines

nanbield as well as kirkstone can be fixed by cherry-picking:
https://git.openembedded.org/bitbake/commit/?id=d01d684a0 asyncrpc: Add context manager API

+ Joshua if he sees some risks in cherry-picking this to stable releases.

While we wait for Joshua to comment I'll do a little autobuilder testing to see if anything blows up with this patch.

FWIW, a-full autobuilder runs for both nanbield and kirkstone with this patch came back all green.

Steve

Quoted 8 lines

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#193373): https://lists.openembedded.org/g/openembedded-core/message/193373
Mute This Topic: https://lists.openembedded.org/mt/102826186/3620601
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#22 lore raw
In reply to Martin Jansa

2024. 01. 05. 17:54 keltezéssel, Martin Jansa írta:

Quoted 202 lines · On Fri, Jan 5, 2024 at 5:06 PM Martin Jansa via lists.openembedded.org…
    FWIW: I see the processes running after bitbake exit only in kirkstone and only with
    PRSERV_HOST = "localhost:0"
    if I drop PRSERV_HOST then all 3 bitbake-server processes exist together with
    bitbake itself.

    luneos-kirkstone $ rm -f bitbake-cookerdaemon.log
    luneos-kirkstone $ ps aux | grep luneos-kirkstone
    martin   4184043  0.0  0.0   9124  2048 pts/4    S+ 16:47   0:00 grep --colour=auto
    luneos-kirkstone
    luneos-kirkstone $ bitbake -k zlib-native
    NOTE: Started PRServer with DBfile: /OE/build/luneos-kirkstone/cache/prserv.sqlite3,
    Address: 127.0.0.1:35811 <http://127.0.0.1:35811>, PID: 4184924
    Loading cache: 100%
    |##########################################################################################################################################################################################################|
    Time: 0:00:04
    Loaded 4570 entries from dependency cache.
    ...
    Sstate summary: Wanted 10 Local 10 Mirrors 0 Missed 0 Current 11 (100% match, 100%
    complete)
    NOTE: Executing Tasks
    NOTE: Tasks Summary: Attempted 92 tasks of which 92 didn't need to be rerun and all
    succeeded.
    NOTE: Writing buildhistory
    NOTE: Writing buildhistory took: 15 seconds
    luneos-kirkstone $ ps aux | grep luneos-kirkstone
    martin     51324  0.0  0.0   9124  2048 pts/4    S+ 16:54   0:00 grep --colour=auto
    luneos-kirkstone
    martin   4184584  8.9  0.4 668196 536892 ?       Sl 16:47   0:37 bitbake-server
    /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server decafbad 3 5
    /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
    /OE/build/luneos-kirkstone/bitbake.lock /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
    martin   4184924  0.0  0.1 340280 196428 ?       S  16:47   0:00 bitbake-server
    /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server decafbad 3 5
    /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
    /OE/build/luneos-kirkstone/bitbake.lock /OE/build/luneos-kirkstone/bitbake.sock 0 None 0
    martin   4184927  0.0  0.1 340328 194440 ?       S  16:47   0:00 bitbake-server
    /OE/build/luneos-kirkstone/bitbake/bin/bitbake-server decafbad 3 5
    /OE/build/luneos-kirkstone/bitbake-cookerdaemon.log
    /OE/build/luneos-kirkstone/bitbake.lock /OE/build/luneos-kirkstone/bitbake.sock 0
    None 0

    luneos-kirkstone $ cat bitbake-cookerdaemon.log
    4184584 16:47:38.367593 --- Starting bitbake server pid 4184584 at 2024-01-05
    16:47:38.367526 ---
    4184584 16:47:38.370315 Started bitbake server pid 4184584
    4184584 16:47:38.370831 Entering server connection loop
    4184584 16:47:38.372043 Accepting [<socket.socket fd=6, family=1, type=1, proto=0,
    laddr=bitbake.sock>] ([])
    4184584 16:47:38.376431 Processing Client
    4184584 16:47:38.376633 Connecting Client
    4184584 16:47:38.377632 Running command ['setFeatures', [2, 1]]
    4184584 16:47:38.379355 Command Completed
    4184584 16:47:38.379868 Running command ['updateConfig', {'halt': False, 'force':
    False, 'invalidate_stamp': None, 'dry_run': False, 'dump_signatures': [],
    'extra_assume_provided': [], 'profile': False, 'prefile': [], 'postfile': [],
    'server_timeout': None, 'nosetscene': False, 'setsceneonly': False, 'skipsetscene':
    False, 'runall': None, 'runonly': None, 'writeeventlog': None,
    'build_verbose_shell': False, 'build_verbose_stdout': False, 'default_loglevel': 20,
    'debug_domains': {}}, {'SHELL': '/bin/bash', 'BB_ENV_PASSTHROUGH_ADDITIONS':
    'MACHINE DISTRO TCMODE TCLIBC http_proxy ftp_proxy https_proxy all_proxy ALL_PROXY
    no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY SDKMACHINE BB_NUMBER_THREADS
    PARALLEL_MAKE GIT_PROXY_COMMAND GIT_PROXY_IGNORE SOCKS5_PASSWD SOCKS5_USER
    WEBOS_DISTRO_BUILD_ID PSEUDO_DISABLED PSEUDO_BUILD', 'DISTRO': 'luneos',
    'SSH_AUTH_SOCK': '/tmp/ssh-XXXXXX2ZxYBo/agent.1893', 'SSH_AGENT_PID': '1894', 'PWD':
    '/OE/build/luneos-kirkstone', 'LOGNAME': 'martin', 'HOME': '/home/martin-gentoo',
    'MACHINE': 'qemux86-64', 'USER': 'martin', 'PATH':
    '/OE/build/luneos-kirkstone/openembedded-core/scripts:/OE/build/luneos-kirkstone/bitbake/bin:/home/martin-gentoo/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/opt/bin:/usr/lib/llvm/17/bin:/opt/android-sdk-update-manager/tools:/opt/android-sdk-update-manager/platform-tools:/etc/eselect/wine/bin:/usr/lib64/opencascade/bin:/var/lib/snapd/snap/bin',
    'LC_ALL': 'en_US.UTF-8', 'SESSION_MANAGER':
    'local/jama:@/tmp/.ICE-unix/1838,unix/jama:/tmp/.ICE-unix/1838', 'WINDOWID':
    '35651587', 'COLORTERM': 'truecolor', 'CSF_MDTVTexturesDirectory':
    '/usr/share/opencascade/resources/Textures', 'MACHINES': 'qemux86-64',
    'XDG_CONFIG_DIRS': '/etc/xdg', 'LESS': '-R -M --shift 5', 'XDG_SESSION_PATH':
    '/org/freedesktop/DisplayManager/Session1', 'XDG_MENU_PREFIX': 'xfce-', 'JDK_HOME':
    '/etc/java-config-2/current-system-vm', 'CONFIG_PROTECT_MASK': '/etc/sandbox.d
    /etc/fonts/fonts.conf /etc/gentoo-release /etc/dconf /etc/ca-certificates.conf
    /etc/revdep-rebuild', 'TERMCAP': 'SC|screen.xterm-256color|VT 100/ANSI X3.64 virtual
    terminal:DO=\\E[%dB:LE=\\E[%dD:RI=\\E[%dC:UP=\\E[%dA:bs:bt=\\E[Z:cd=\\E[J:ce=\\E[K:cl=\\E[H\\E[J:cm=\\E[%i%d;%dH:ct=\\E[3g:do=^J:nd=\\E[C:pt:rc=\\E8:rs=\\Ec:sc=\\E7:st=\\EH:up=\\EM:le=^H:bl=^G:cr=^M:it#8:ho=\\E[H:nw=\\EE:ta=^I:is=\\E)0:li#64:co#238:am:xn:xv:LP:sr=\\EM:al=\\E[L:AL=\\E[%dL:cs=\\E[%i%d;%dr:dl=\\E[M:DL=\\E[%dM:dc=\\E[P:DC=\\E[%dP:im=\\E[4h:ei=\\E[4l:mi:IC=\\E[%d@:ks=\\E[?1h\\E=:ke=\\E[?1l\\E>:vi=\\E[?25l:ve=\\E[34h\\E[?25h:vs=\\E[34l:ti=\\E[?1049h:te=\\E[?1049l:us=\\E[4m:ue=\\E[24m:so=\\E[3m:se=\\E[23m:mb=\\E[5m:md=\\E[1m:mh=\\E[2m:mr=\\E[7m:me=\\E[m:ms:Co#8:pa#64:AF=\\E[3%dm:AB=\\E[4%dm:op=\\E[39;49m:AX:vb=\\Eg:G0:as=\\E(0:ae=\\E(B:ac=\\140\\140aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~..--++,,hhII00:po=\\E[5i:pf=\\E[4i:Z0=\\E[?3h:Z1=\\E[?3l:Km=\\E[<:k0=\\E[10~:k1=\\EOP:k2=\\EOQ:k3=\\EOR:k4=\\EOS:k5=\\E[15~:k6=\\E[17~:k7=\\E[18~:k8=\\E[19~:k9=\\E[20~:k;=\\E[21~:F1=\\E[23~:F2=\\E[24~:kB=\\E[Z:kh=\\E[1~:@1=\\E[1~:kH=\\E[4~:@7=\\E[4~:kN=\\E[6~:kP=\\E[5~:kI=\\E[2~:kD=\\E[3~:ku=\\EOA:kd=\\EOB:kr=\\EOC:kl=\\EOD:km:',
    'JAVA_HOME': '/etc/java-config-2/current-system-vm', 'CSF_MIGRATION_TYPES':
    '/usr/share/opencascade/resources/StdResource/MigrationSheet.txt', 'WINDOW': '3',
    'ANT_HOME': '/usr/share/ant', 'DESKTOP_SESSION': 'xfce', 'CSF_STEPDefaults':
    '/usr/share/opencascade/resources/XSTEPResource', 'EDITOR': 'vim', 'GTK_MODULES':
    'canberra-gtk-module', 'XDG_SEAT': 'seat0', 'CONFIG_PROTECT': '/var/bind
    /usr/share/gnupg/qualified.txt /usr/share/easy-rsa /usr/share/config
    /usr/lib64/libreoffice/program/sofficerc', 'XDG_SESSION_DESKTOP': 'XFCE',
    'XDG_SESSION_TYPE': 'x11', 'PANEL_GDK_CORE_DEVICE_EVENTS': '0', 'MANPATH':
    '/etc/java-config-2/current-system-vm/man:/usr/share/gcc-data/x86_64-pc-linux-gnu/13/man:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.41/man:/etc/java-config-2/current-system-vm/man/:/usr/local/share/man:/usr/share/man:/usr/lib/rust/man:/usr/lib/llvm/17/share/man:/etc/eselect/wine/share/man',
    'XAUTHORITY': '/tmp/xauth_CEynPY', 'CSF_StandardLiteDefaults':
    '/usr/share/opencascade/resources/StdResource', 'OPENCL_PROFILE': 'nvidia',
    'MOTD_SHOWN': 'pam', 'GDM_LANG': 'C.UTF-8', 'LANG': 'en_US.UTF-8', 'LS_COLORS':
    'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=01;05;37;41:mi=01;05;37;41:su=37;41:sg=30;43:ca=00:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.avif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.cfg=00;32:*.conf=00;32:*.diff=00;32:*.doc=00;32:*.ini=00;32:*.log=00;32:*.patch=00;32:*.pdf=00;32:*.ps=00;32:*.tex=00;32:*.txt=00;32:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:*~=00;90:*#=00;90:*.bak=00;90:*.crdownload=00;90:*.dpkg-dist=00;90:*.dpkg-new=00;90:*.dpkg-old=00;90:*.dpkg-tmp=00;90:*.old=00;90:*.orig=00;90:*.part=00;90:*.rej=00;90:*.rpmnew=00;90:*.rpmorig=00;90:*.rpmsave=00;90:*.swp=00;90:*.tmp=00;90:*.ucf-dist=00;90:*.ucf-new=00;90:*.ucf-old=00;90:',
    'XDG_CURRENT_DESKTOP': 'XFCE', 'VTE_VERSION': '7402', 'CSF_GraphicShr':
    '/usr/lib64/opencascade/libTKOpenGl.so', 'OPENGL_PROFILE': 'xorg-x11',
    'XDG_SEAT_PATH': '/org/freedesktop/DisplayManager/Seat0', 'QT_GRAPHICSSYSTEM':
    'raster', 'XZ_OPT': '-T0', 'CSF_ShadersDirectory':
    '/usr/share/opencascade/resources/Shaders', 'CSF_XmlOcafResource':
    '/usr/share/opencascade/resources/XmlOcafResource', 'INFOPATH':
    '/usr/share/gcc-data/x86_64-pc-linux-gnu/13/info:/usr/share/binutils-data/x86_64-pc-linux-gnu/2.41/info:/usr/share/autoconf-2.72/info:/usr/share/autoconf-2.13/info:/usr/share/automake-1.16.5/info:/usr/share/info',
    'MOZ_GMP_PATH': '/usr/lib64/nsbrowser/plugins/gmp-gmpopenh264/system-installed',
    'CSF_SHMessage': '/usr/share/opencascade/resources/SHMessage', 'JAVAC':
    '/etc/java-config-2/current-system-vm/bin/javac', 'XDG_SESSION_CLASS': 'user',
    'VBOX_SDK_PATH': '/usr/lib64/virtualbox/sdk', 'ANDROID_HOME':
    '/opt/android-sdk-update-manager', 'TERM': 'screen.xterm-256color',
    'VBOX_INSTALL_PATH': '/usr/lib64/virtualbox', 'LESSOPEN': '|lesspipe %s',
    'ANDROID_SWT': '/usr/share/swt-3.7/lib', 'CSF_StandardDefaults':
    '/usr/share/opencascade/resources/StdResource', 'CSF_IGESDefaults':
    '/usr/share/opencascade/resources/XSTEPResource', 'MANPAGER': 'manpager',
    'NPM_CONFIG_GLOBALCONFIG': '/etc/npm/npmrc', 'DISPLAY': ':0.0', 'CSF_XCAFDefaults':
    '/usr/share/opencascade/resources/StdResource', 'SHLVL': '2', 'PAGER':
    '/usr/bin/less', 'LEX': 'flex', 'XDG_VTNR': '2', 'CSF_PluginDefaults':
    '/usr/share/opencascade/resources/StdResource', 'XDG_SESSION_ID': '2', 'CASROOT':
    '/usr', 'XDG_RUNTIME_DIR': '/run/user/1026', 'GCC_SPECS': '', 'GSETTINGS_BACKEND':
    'dconf', 'PS1': '\\[\\033[01;32m\\]OE $MACHINE@$DISTRO\\[\\033[01;34m\\] \\w
    \\$\\[\\033[00m\\] ', 'CSF_XSMessage': '/usr/share/opencascade/resources/XSMessage',
    'XDG_DATA_DIRS':
    '/usr/local/share:/usr/share:/etc/eselect/wine/share:/var/lib/snapd/desktop',
    'CSF_UnitsDefinition': '/usr/share/opencascade/resources/UnitsAPI/Units.dat', 'STY':
    '2067.x', 'VBOX_APP_HOME': '/usr/lib64/virtualbox', 'DBUS_SESSION_BUS_ADDRESS':
    'unix:path=/run/user/1026/bus', 'INITPWD': '/OE/build/luneos-kirkstone', 'MAIL':
    '/var/mail/martin', 'OLDPWD': '/home/martin-gentoo', '_':
    '/OE/build/luneos-kirkstone/bitbake/bin/bitbake'},
    ['/OE/build/luneos-kirkstone/bitbake/bin/bitbake', '-k', 'zlib-native']]
    /OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228: DeprecationWarning:
    ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
      if isinstance(node.args[0], ast.Str):
    /OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229: DeprecationWarning:
    Attribute s is deprecated and will be removed in Python 3.14; use value instead
      varname = node.args[0].s
    /OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:230: DeprecationWarning:
    ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
      if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
    /OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:233: DeprecationWarning:
    Attribute s is deprecated and will be removed in Python 3.14; use value instead
      self.contains[varname].add(node.args[1].s)
    /usr/lib/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning: This
    process (pid=4184584) is multi-threaded, use of fork() may lead to deadlocks in the
    child.
      self.pid = os.fork()
    4184584 16:47:45.108449 Command Completed
    4184584 16:47:45.112267 Running command ['getVariable', 'BBINCLUDELOGS']
    4184584 16:47:45.113080 Command Completed
    4184584 16:47:45.113399 Running command ['getVariable', 'BBINCLUDELOGS_LINES']
    4184584 16:47:45.113925 Command Completed
    4184584 16:47:45.114218 Running command ['getSetVariable', 'BB_CONSOLELOG']
    4184584 16:47:45.114956 Command Completed
    4184584 16:47:45.115441 Running command ['getSetVariable', 'BB_LOGCONFIG']
    4184584 16:47:45.116590 Command Completed
    4184584 16:47:45.122232 Running command ['getUIHandlerNum']
    4184584 16:47:45.122805 Command Completed
    4184584 16:47:45.123194 Running command ['setEventMask', 1, 20,
    {'BitBake.SigGen.HashEquiv': 19, 'BitBake.RunQueue.HashEquiv': 19},
    ['bb.runqueue.runQueueExitWait', 'bb.event.LogExecTTY', 'logging.LogRecord',
    'bb.build.TaskFailed', 'bb.build.TaskBase', 'bb.event.ParseStarted',
    'bb.event.ParseProgress', 'bb.event.ParseCompleted', 'bb.event.CacheLoadStarted',
    'bb.event.CacheLoadProgress', 'bb.event.CacheLoadCompleted',
    'bb.command.CommandFailed', 'bb.command.CommandExit', 'bb.command.CommandCompleted',
    'bb.cooker.CookerExit', 'bb.event.MultipleProviders', 'bb.event.NoProvider',
    'bb.runqueue.sceneQueueTaskStarted', 'bb.runqueue.runQueueTaskStarted',
    'bb.runqueue.runQueueTaskFailed', 'bb.runqueue.sceneQueueTaskFailed',
    'bb.event.BuildBase', 'bb.build.TaskStarted', 'bb.build.TaskSucceeded',
    'bb.build.TaskFailedSilent', 'bb.build.TaskProgress', 'bb.event.ProcessStarted',
    'bb.event.ProcessProgress', 'bb.event.ProcessFinished']]
    4184584 16:47:45.123558 Command Completed
    4184584 16:47:45.123863 Running command ['getVariable', 'BB_DEFAULT_TASK']
    4184584 16:47:45.124228 Command Completed
    4184584 16:47:45.124528 Running command ['setConfig', 'cmd', 'build']
    4184584 16:47:45.124858 Command Completed
    4184584 16:47:45.125140 Running command ['buildTargets', ['zlib-native'], 'build']
    4184584 16:47:45.125294 Command Completed
    /OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228: DeprecationWarning:
    ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
      if isinstance(node.args[0], ast.Str):
    /OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229: DeprecationWarning:
    Attribute s is deprecated and will be removed in Python 3.14; use value instead
      varname = node.args[0].s
    /OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:230: DeprecationWarning:
    ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
      if name in self.containsfuncs and isinstance(node.args[1], ast.Str):
    /OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:233: DeprecationWarning:
    Attribute s is deprecated and will be removed in Python 3.14; use value instead
      self.contains[varname].add(node.args[1].s)
    /usr/lib/python3.12/asyncio/streams.py:413: ResourceWarning: unclosed <StreamWriter
    transport=<_SelectorSocketTransport closing fd=18 read=idle write=<idle, bufsize=0>>
    reader=<StreamReader transport=<_SelectorSocketTransport closing fd=18 read=idle
    write=<idle, bufsize=0>>>>
      warnings.warn(f"unclosed {self!r}", ResourceWarning)
    /OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:228: DeprecationWarning:
    ast.Str is deprecated and will be removed in Python 3.14; use ast.Constant instead
      if isinstance(node.args[0], ast.Str):
    /OE/build/luneos-kirkstone/bitbake/lib/bb/codeparser.py:229: DeprecationWarning:
    Attribute s is deprecated and will be removed in Python 3.14; use value instead
      varname = node.args[0].s
    4184584 16:48:37.909973 Processing Client
    4184584 16:48:37.910125 Disconnecting Client
    4184584 16:48:37.910729 No timeout, exiting.
    4184584 16:48:38.011341 Exiting

    It is reproducible with nanbield as well and fixed in scarthgap, will bisect it when
    time allows.
Quoted 4 lines

bisect was simpler than expected as nanbield is still compatible with newer bitbake and there weren't that many changes.

nanbield as well as kirkstone can be fixed by cherry-picking:
https://git.openembedded.org/bitbake/commit/?id=d01d684a0 asyncrpc: Add context manager API

I have tested it by cherry-picking this commit into bitbake in nanbield and there are no stuck bitbake-server processes with this change under Fedora 39.

Quoted 4 lines

+ Joshua if he sees some risks in cherry-picking this to stable releases.

Cheers