Skip to contentSkip to conversation

nativesdk-libgcc-initial race condition and sstate-mirror problems

DISCUSSION Dimitar Yordanov 1 participant 1 message Jul 31, 2015

lore
Share & tools
lore
#0 lore raw

Hi all,

we are experiencing problems with our sstate-mirror handling after applying commit: be02588097dcdb61b76e5788a67ab062dabc78ad

Here is a short explanation to what we do and what happens:

We have an sstate-mirror, which we want to keep up to date when new builds are executed and also be clean from unused sstate files. For this purpose we set the SSTATE_MIRROR variable to point to the latest stable cache we have. Then we trigger a new build with changes to some of the recipes/sources. Under SSTATE_DIR a new cache is being created which has links to SSTATE_MIRROR for caches, which are still valid.

To get a new clean SSTATE_MIRROR we take the newly created SSTATE_DIR and replace all the links with the real files. Then we promote that SSTATE_DIR to be the new SSTATE_MIRROR.

Where does the problem arises ...

With the previously mentioned patch the do_package tasks are being skipped, thus there are no entries or links for that in the new SSTATE_DIR. When we now promote the SSTATE_DIR to a SSTATE_MIRROR bitbake decides that the do_package tasks need to be rebuilt, because the squashed reverse dependencies are not covered (runqueue.py:1931->execute). setscene_depvalid does not even get called.

Maybe we wouldn't have noticed that, but we get some wired race conditions because of that, between nativesdk-libgcc-initial, nativesdk-glibc and nativesdk-linux-libc-headers. What happens:

All packages have an sstate cache entries (apart from do_package), and they get executed, which populates the nativesdk sysroot with limits.h and linux/limits.h

And now bitbake decides it should re-trigger the do_configure stage of nativesdk-linux-libc-headers. That removes linux/limits.h, but limits.h is still there and includes (indirect) linux/limits.h. In this moment the do_configure step from nativesdk-libgcc-initial is called, which includes limits.h. Normally if the file would've not be there it would have used the embedded one from gcc but now there is a file there and it uses it. The bad point is limits.h indirectly includes linux/limits.h and the do_configure step from nativesdk-libgcc-initial fails :-(

I see multiple issues here and would be glad to get your opinion on how to solve them properly:
First problem skipped tasks and missing do_package entries in the sstate. Second the race condition with nativesdk-libgcc-initial.

Best wishes
Dimitar

Signature · 13 lines
BMW Car IT GmbH
Dimitar Yordanov
Lise-Meitner-Str.14
89081 Ulm
Tel.: +49-731-37804-290
Fax: +49-731 378040-01
Mail: [email protected]
Web: http://www.bmw-carit.de
-------------------------------------------------------------------------
BMW Car IT GmbH
Geschäftsführer: Reinhard Stolle und Michael Würtenberger
Sitz und Registergericht: München HRB 134810
-------------------------------------------------------------------------