PSA for anyone using openSUSE 42.2 (or similar)
Lately I've been seeing very intermittent, sporadic cases where a build will fail with something similar to:
fork: Resource temporarily unavailable
and nothing else wrong. Simply restarting the build causes it to succeed without incident. I had been seeing this, maybe, twice a week (on a build machine that builds most of the day as well as overnight with Jenkins jobs).
I decided to investigate and found:
which suggests the issue is a combination of
- linux >= 4.3
- systemd >= 228
both of which are true when upgrading openSUSE from 42.1 (linux-4.1.36-41/210) to 42.2 (linux-4.4.36-8/228).
The fix appears to be to modify the "DefaultTasksMax" systemd parameter (uncommenting it and setting it to "infinity"). On openSUSE 42.2 this is found in /etc/systemd/system.conf, then
# systemctl daemon-reexec
or reboot.
I just made this adjustment but given it only happened once or twice a week on a reasonably busy build machine it might be hard to confirm. I'm also not sure if setting it to "infinity" is sensible ;-)
I'm guessing other host distros probably also fall into this category.
Quoted 5 lines · Trevor Woerner
I just made this adjustment but given it only happened once or twice a week on a reasonably busy build machine it might be hard to confirm. I'm also not sure if setting it to "infinity" is sensible ;-)
I'm guessing other host distros probably also fall into this category.
I've seen this on Debian Testing as well. Care to submit a documentation bug?
https://bugzilla.yoctoproject.org/enter_bug.cgi?product=General%20Docs
Alex
Quoted 7 lines · Alexander Kanavin
Quoted 6 lines · Trevor Woerner
I just made this adjustment but given it only happened once or twice a week on
a reasonably busy build machine it might be hard to confirm. I'm also not sure
if setting it to "infinity" is sensible ;-)
Quoted 6 lines
Quoted 1 line
I'm guessing other host distros probably also fall into this category.
I've seen this on Debian Testing as well. Care to submit a documentation bug?
https://bugzilla.yoctoproject.org/enter_bug.cgi?product=General%20Docs
Actually, scratch that; if bitbake can detect that the limit is not optimal at runtime, then it probably should. Direct the bug accordingly.
Alex
Quoted 2 lines · Alexander Kanavin
Actually, scratch that; if bitbake can detect that the limit is not optimal at runtime, then it probably should. Direct the bug accordingly.
Bitbake can't possibly know how many builds you're going to spin up at any one time, and it can't take into consideration whether of not you set "parallel build" or "number of threads" settings on any of those builds. So when each bitbake starts it might look at the variable and say "that looks reasonable" but after starting the 3rd bitbake you end up with resource issues that fail a build that might otherwise muddle through.
Quoted 8 lines · Alexander Kanavin
Quoted 7 lines · Trevor Woerner
I just made this adjustment but given it only happened once or twice a week on
a reasonably busy build machine it might be hard to confirm. I'm also not sure
if setting it to "infinity" is sensible ;-)
Quoted 7 lines
Quoted 1 line
I'm guessing other host distros probably also fall into this category.
I've seen this on Debian Testing as well. Care to submit a documentation bug?
https://bugzilla.yoctoproject.org/enter_bug.cgi?product=General%20Docs