Deterministic build problem
Hi,
There are several recipes have broken deterministic build problem.
wget/quilt/tmon/vim/vim-tiny
Here is how to reproduce it:
bitbake recipename
add "bb.note('test')" in do_fetch of base.bbclass
bitbake recipename
we can find for above recipes the installed files under ${WORKDIR}/image is different
since the different configure between 2 times' build. And the differences always caused
by different NATIVE_SYSROOT, the second time build will have more native binaries installed
during the first time build, and influence the second time's configure task.
eg:
wget: do_package installed rpm-native
quilt: do_patch installed quilt-native
vim: do_package_write_rpm installeddd glib-2.0-native
tmon: do_install installed pkgconfig-native
I know we can fix per recipe by patch configure/Makefile or change PACKAGECONFIG etc, but this
seems a common problem, is it possible fix this in common framework? Thanks.
Quoted 38 lines · Changqing Li
There are several recipes have broken deterministic build problem.
wget/quilt/tmon/vim/vim-tiny
Here is how to reproduce it:
bitbake recipename
add "
bb.note('test')" indo_fetchof base.bbclassbitbake recipename
we can find for above recipes the installed files under ${WORKDIR}/image
is differentsince the different configure between 2 times' build. And the differences always caused
by different
NATIVE_SYSROOT, the second time build will have more native binaries installedduring the first time build, and influence the second time's configure
task.eg:
wget:
do_packageinstalled rpm-nativequilt:
do_patchinstalled quilt-nativevim:
do_package_write_rpminstalleddd glib-2.0-nativetmon:
do_installinstalled pkgconfig-native
Quoted 6 lines
I know we can fix per recipe by patch configure/Makefile or change PACKAGECONFIG etc, but this
seems a common problem, is it possible fix this in common framework?
Thanks.
Its a good question, one I've pondered a lot. I realised the issue as we were finishing implementing recipe specific sysroots. The alternative, task based sysroots are impractical as you have to reconfigure the source between tasks which defeats the objective.
The challenge is knowing when its safe to remove things from the sysroot and when it is not. For example, do_package can run in parallel with do_populate_sysroot and one needs rpm-native, the other does not. Should do_populate_sysroot uninstall rpm-native? No, do_package may be using it.
There are some serial points where it might be safe to cleanup, e.g. do_configure is probably ok. We've not done that as its hard to be sure and its not generic code.
We could force all tasks to run serially, but that does seem a little sad to lose the parallelism when it generally works well.
I think we probably can improve, not sure we can fix it for all cases though generically without significant performance/flexibility loss.
Cheers,
Richard