nativesdk-libgcc-initial race condition and sstate-mirror problems
Hi all,
we are experiencing problems with our sstate-mirror handling after applying commit: be02588097dcdb61b76e5788a67ab062dabc78ad
Here is a short explanation to what we do and what happens:
We have an sstate-mirror, which we want to keep up to date when new builds are executed and also be clean from unused sstate files. For this purpose we set the SSTATE_MIRROR variable to point to the latest stable cache we have. Then we trigger a new build with changes to some of the recipes/sources. Under SSTATE_DIR a new cache is being created which has links to SSTATE_MIRROR for caches, which are still valid.
To get a new clean SSTATE_MIRROR we take the newly created SSTATE_DIR and replace all the links with the real files. Then we promote that SSTATE_DIR to be the new SSTATE_MIRROR.
Where does the problem arises ...
With the previously mentioned patch the do_package tasks are being skipped, thus there are no entries or links for that in the new SSTATE_DIR. When we now promote the SSTATE_DIR to a SSTATE_MIRROR bitbake decides that the do_package tasks need to be rebuilt, because the squashed reverse dependencies are not covered (runqueue.py:1931->execute). setscene_depvalid does not even get called.
Maybe we wouldn't have noticed that, but we get some wired race conditions because of that, between nativesdk-libgcc-initial, nativesdk-glibc and nativesdk-linux-libc-headers. What happens:
All packages have an sstate cache entries (apart from do_package), and they get executed, which populates the nativesdk sysroot with limits.h and linux/limits.h
And now bitbake decides it should re-trigger the do_configure stage of nativesdk-linux-libc-headers. That removes linux/limits.h, but limits.h is still there and includes (indirect) linux/limits.h. In this moment the do_configure step from nativesdk-libgcc-initial is called, which includes limits.h. Normally if the file would've not be there it would have used the embedded one from gcc but now there is a file there and it uses it. The bad point is limits.h indirectly includes linux/limits.h and the do_configure step from nativesdk-libgcc-initial fails :-(
I see multiple issues here and would be glad to get your opinion on how to solve them properly:
First problem skipped tasks and missing do_package entries in the sstate. Second the race condition with nativesdk-libgcc-initial.
Best wishes
Dimitar
Signature · 13 lines
BMW Car IT GmbH Dimitar Yordanov Lise-Meitner-Str.14 89081 Ulm Tel.: +49-731-37804-290 Fax: +49-731 378040-01 Mail: [email protected] Web: http://www.bmw-carit.de ------------------------------------------------------------------------- BMW Car IT GmbH Geschäftsführer: Reinhard Stolle und Michael Würtenberger Sitz und Registergericht: München HRB 134810 -------------------------------------------------------------------------