Re-execution of tasks - test report and results

DISCUSSION Richard Purdie 4 participants 9 messages Mar 31, 2012 – Apr 6, 2012

Share & tools

Richard Purdie

author

2012-03-31 14:16 UTC #0 lore raw

As some people have noticed, there are some rebuild issues happening due to sstate and the use of hashes in the stamp files. By this I mean the case where due to some checksum change, some task gets rerun and the task was not written to run a second time.

In other words all tasks are not idempotent (thanks Koen!) but should be.

For the purposes of finding these tasks, we have the open bug:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=2123

and there are two proposed scripts there. One is a simple forced re-execution of each task in turn. This catches some issues but not others. I therefore wrote a second slower script which after forcing a task, re-executes the current target to completion. The second script is much slower than the first but finds different errors and better pinpoints some others.

I've had my build machine iterating with the second script for a while and its tested about 5,000 task re-executions and identified a number of failures. Its not complete yet but its mostly there and I'm going to put the failures in two groups:

a) Don't build at all:

alsa-tools.compile - need to check into/fix (local issue, works on AB?) insserv-native.compile - need to check into/fix or delete libx11-diet.compile - need to check into/fix external-python-tarball - need to check into/fix external-poky-toolchain - time to delete this recipe? package-index - rpm package feed generation dependencies missing, has open bug gobject-introspection - Known issue, not cross compile capable

Most of these are things excluded from world which have "fallen through the cracks" or are known issues.

b) Failures in specific task re-execution

boost.boostconfig
boost.patch
docbook-utils-native.unpack
dropbear.debug_patch
eglibc-initial-nativesdk.patch
eglibc-initial.patch
eglibc-nativesdk.patch
eglibc.patch
gcc.configure
gcc-cross-canadian-i586.configure
gcc-cross-canadian-i586.headerfix
gcc-cross-canadian-i586.patch
gcc-cross-canadian-i586.unpack
gcc.headerfix
gcc.patch
gcc.unpack
man-pages.unpack
nasm-native.patch
nasm-native.patch_fixaclocal
nasm.patch
nasm.patch_fixaclocal
net-tools.patch
net-tools.unpack
perl.patch
python-native.patch
python-nativesdk.patch
python.patch
qt-x11-free.configure
qt-x11-free.generate_qt_config_file
qt-x11-free.patch
sgml-common-native.compile
unfs-server-native.configure
unfs-server-nativesdk.configure
wget.patch

To reproduce, just run "bitbake xxx -c cleansstate; bitbake xxx; bitbake xxx -c yyy -f; bitbake xxx".

I've weeded out the false positives which were things like errors about multiple providers changing bitbake's exit code. I also found building libiconv totally destroyed the sysroot and caused iconv.h failures so I blacklisted it.

Is there anything these tests won't find?

Sadly, yes :(

If you do something like "bitbake -c compile perl -f; bitbake git -c compile -f", it breaks since there is a dependency there with the timestamps that causes problems. Neither script above would conclusively detect this, you might get lucky with the first one.

Secondly, in these tests we didn't check "does the output change?" since we have no good tool to do this yet.

I'd propose we at least get the above issues identified fixed. People can then report any other issues they run into and we fix them as we find them...

We also need to go through Jiajun's list in the bugzilla carefully too since I think there are some different issues being exposed there. Some are duplicates of the above, some are not.

Cheers,

Richard

Steve Sakoman

2012-03-31 15:14 UTC #1 lore raw

Quoted 4 lines · Richard Purdie

As some people have noticed, there are some rebuild issues happening due to sstate and the use of hashes in the stamp files. By this I mean the case where due to some checksum change, some task gets rerun and the task was not written to run a second time.

For the set of recipes I typically build with my autobuild script I found it necessary to add code to do a -c cleansstate on the following recipes:

shadow-native
boost
gcc
git
man-pages
net-tools
sqlite
wget

There seems to be a good amount of overlap with your list, but there are a few that are not on your list.

Steve

Richard Purdie

author

2012-03-31 15:43 UTC #2 lore raw

Quoted 20 lines · Steve Sakoman

Quoted 4 lines · Richard Purdie
As some people have noticed, there are some rebuild issues happening due to sstate and the use of hashes in the stamp files. By this I mean the case where due to some checksum change, some task gets rerun and the task was not written to run a second time.
For the set of recipes I typically build with my autobuild script I found it necessary to add code to do a -c cleansstate on the following recipes:
shadow-native
boost
gcc
git
man-pages
net-tools
sqlite
wget

Quoted 2 lines

There seems to be a good amount of overlap with your list, but there are a few that are not on your list.

Hmm, yes. We already have a bug open against git but I'd like to understand shadow-native and sqlite. If you have the failure logs for those two, I think it would be worth opening bugs for them.

Cheers,

Richard

Steve Sakoman

2012-03-31 16:08 UTC #3 lore raw

Quoted 3 lines · Richard Purdie

Hmm, yes. We already have a bug open against git but I'd like to understand shadow-native and sqlite. If you have the failure logs for those two, I think it would be worth opening bugs for them.

Details for shadow-native and sqlite below:

shadow-native:

ERROR: Function failed: do_configure (see /media/data/yocto/tmp/work/x86_64-linux/shadow-native-4.1.4.3-r3/temp/log.do_configure.23277 for further information)
ERROR: Logfile of failure stored in:
/media/data/yocto/tmp/work/x86_64-linux/shadow-native-4.1.4.3-r3/temp/log.do_configure.23277 Log data follows:
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/grpck
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/usermod
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/logoutd
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/vigr
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/nologin
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/chgpasswd
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/grpunconv
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/pwconv
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/groupmems
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/pwunconv
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/groupadd
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/groupdel
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/useradd
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/groupmod
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/grpconv
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/pwck
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/chpasswd.shadow-native
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/newusers
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/userdel
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/vipw
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/chage
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/newgrp.shadow-native
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/passwd.shadow-native
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/groups
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/su
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/chfn.shadow-native
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/expiry
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/gpasswd
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/sg
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/login
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/faillog
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/lastlog
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/chsh.shadow-native
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man3/getspnam.3
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man3/shadow.3
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man1/chage.1
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man1/su.1
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man1/gpasswd.1
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man1/login.1
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man1/expiry.1
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man1/chsh.1
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man1/groups.1
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man1/newgrp.1
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man1/sg.1
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man1/passwd.1
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man1/chfn.1
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/chgpasswd.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/lastlog.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/groupdel.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/groupadd.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/pwck.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/vigr.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/grpunconv.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/pwunconv.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/usermod.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/newusers.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/nologin.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/logoutd.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/grpconv.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/chpasswd.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/useradd.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/pwconv.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/grpck.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/userdel.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/faillog.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/groupmems.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/vipw.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/groupmod.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man5/passwd.5
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man5/limits.5
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man5/gshadow.5
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man5/login.access.5 | DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man5/shadow.5
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man5/porttime.5
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man5/login.defs.5
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man5/suauth.5
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man5/faillog.5
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/etc/login.defs
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/etc/login.access
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/etc/limits
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/etc/default/useradd
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man5/
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man8/
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man1/
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/man3/
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/man/
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/etc/default/
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/sbin/

| DEBUG: Removing manifest: /media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/
| DEBUG: Removing manifest: /media/data/yocto/tmp/sysroots/x86_64-linux/sbin/
| DEBUG: Removing manifest: /media/data/yocto/tmp/sysroots/x86_64-linux/etc/
| DEBUG: Removing manifest: /media/data/yocto/tmp/sysroots/x86_64-linux/usr/
| DEBUG: Removing manifest: /media/data/yocto/tmp/sysroots/x86_64-linux/bin/
| ERROR: Function failed: do_configure (see
/media/data/yocto/tmp/work/x86_64-linux/shadow-native-4.1.4.3-r3/temp/log.do_configure.23277
for further information)
| automake (GNU automake) 1.11.2
| Copyright (C) 2011 Free Software Foundation, Inc.
| License GPLv2+: GNU GPL version 2 or later
<http://gnu.org/licenses/gpl-2.0.html>
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
|
| Written by Tom Tromey <[email protected]>
|        and Alexandre Duret-Lutz <[email protected]>.
| AUTOV is 1.11
| NOTE: Executing autoreconf --verbose --install --force

--exclude=autopoint -I/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/aclocal-1.11
-I /media/data/yocto/tmp/work/x86_64-linux/shadow-native-4.1.4.3-r3/shadow-4.1.4.3/aclocal-copy/ | autoreconf: Entering directory `.'
| autoreconf: running: aclocal -I /media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/aclocal-1.11 -I /media/data/yocto/tmp/work/x86_64-linux/shadow-native-4.1.4.3-r3/shadow-4.1.4.3/aclocal-copy/ -I /media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/aclocal-1.11
-I /media/data/yocto/tmp/work/x86_64-linux/shadow-native-4.1.4.3-r3/shadow-4.1.4.3/aclocal-copy/ --force --warnings=cross

| aclocal: unknown warning category `cross'
| configure.in:496: warning: macro `AM_GNU_GETTEXT_VERSION' not found in library
| configure.in:497: warning: macro `AM_GNU_GETTEXT' not found in library
| autoreconf: configure.in: tracing
| autoreconf: configure.in: AM_GNU_GETTEXT_VERSION is used, but not
AM_GNU_GETTEXT
| autoreconf: running: libtoolize --copy --force
| libtoolize: putting auxiliary files in `.'.
| libtoolize: copying file `./ltmain.sh'
| libtoolize: Consider adding `AC_CONFIG_MACRO_DIR([m4])' to configure.in and
| libtoolize: rerunning libtoolize, to keep the correct libtool macros in-tree.
| libtoolize: Consider adding `-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
| aclocal: unknown warning category `cross'
| configure.in:496: warning: macro `AM_GNU_GETTEXT_VERSION' not found in library
| configure.in:497: warning: macro `AM_GNU_GETTEXT' not found in library
| autoreconf: running:

/media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/autoconf --include=/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/aclocal-1.11 --include=/media/data/yocto/tmp/work/x86_64-linux/shadow-native-4.1.4.3-r3/shadow-4.1.4.3/aclocal-copy/ --force --warnings=cross

| configure.in:496: error: possibly undefined macro: AM_GNU_GETTEXT_VERSION
|       If this token and others are legitimate, please use m4_pattern_allow.
|       See the Autoconf documentation.
| configure.in:497: error: possibly undefined macro: AM_GNU_GETTEXT
| autoreconf: /media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/autoconf

failed with exit status: 1
| ERROR: autoreconf execution failed.
NOTE: package shadow-native-4.1.4.3-r3: task do_configure: Failed
ERROR: Task 1316 (/home/steve/source/yocto/poky/meta/recipes-extended/shadow/shadow-native_4.1.4.3.bb, do_configure) failed with exit code '1'

sqlite:

ERROR: Function failed: do_configure (see /media/data/yocto/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/sqlite-2.8.17-r7/temp/log.do_configure.20769 for further information)
ERROR: Logfile of failure stored in:
/media/data/yocto/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/sqlite-2.8.17-r7/temp/log.do_configure.20769 Log data follows:
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/omap3-multi/usr/include/sqlite.h
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/omap3-multi/usr/lib/libsqlite.so.0
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/omap3-multi/usr/lib/libsqlite.so.0.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/omap3-multi/usr/lib/libsqlite.so.0.8.6
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/omap3-multi/usr/lib/libsqlite.so
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/omap3-multi/usr/lib/pkgconfig/sqlite.pc
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/omap3-multi/usr/lib/pkgconfig/
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/omap3-multi/usr/include/

| DEBUG: Removing manifest: /media/data/yocto/tmp/sysroots/omap3-multi/usr/lib/
| DEBUG: Removing manifest: /media/data/yocto/tmp/sysroots/omap3-multi/usr/
| DEBUG: SITE files ['endian-little', 'bit-32', 'arm-common',

'common-linux', 'common-glibc', 'arm-linux', 'arm-linux-gnueabi',
'common']
| ERROR: Function failed: do_configure (see /media/data/yocto/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/sqlite-2.8.17-r7/temp/log.do_configure.20769 for further information)
| main.mk is patched, no need to configure
| mv: cannot stat `/media/data/yocto/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/sqlite-2.8.17-r7/sqlite.pc':
No such file or directory
NOTE: package sqlite-2.8.17-r7: task do_configure: Failed
ERROR: Task 1673 (/home/steve/source/yocto/poky/meta-openembedded/meta-oe/recipes-support/sqlite/sqlite_2.8.17.bb, do_configure) failed with exit code '1'

Richard Purdie

author

2012-03-31 17:04 UTC #4 lore raw

Quoted 36 lines · Steve Sakoman

Quoted 3 lines · Richard Purdie
Hmm, yes. We already have a bug open against git but I'd like to understand shadow-native and sqlite. If you have the failure logs for those two, I think it would be worth opening bugs for them.
Details for shadow-native and sqlite below:
shadow-native:
| aclocal: unknown warning category `cross'
| configure.in:496: warning: macro `AM_GNU_GETTEXT_VERSION' not found in library
| configure.in:497: warning: macro `AM_GNU_GETTEXT' not found in library
| autoreconf: configure.in: tracing
| autoreconf: configure.in: AM_GNU_GETTEXT_VERSION is used, but not
AM_GNU_GETTEXT
| autoreconf: running: libtoolize --copy --force
| libtoolize: putting auxiliary files in `.'.
| libtoolize: copying file `./ltmain.sh'
| libtoolize: Consider adding `AC_CONFIG_MACRO_DIR([m4])' to configure.in and
| libtoolize: rerunning libtoolize, to keep the correct libtool macros in-tree.
| libtoolize: Consider adding `-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
| aclocal: unknown warning category `cross'
| configure.in:496: warning: macro `AM_GNU_GETTEXT_VERSION' not found in library
| configure.in:497: warning: macro `AM_GNU_GETTEXT' not found in library
| autoreconf: running:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/autoconf --include=/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/aclocal-1.11 --include=/media/data/yocto/tmp/work/x86_64-linux/shadow-native-4.1.4.3-r3/shadow-4.1.4.3/aclocal-copy/ --force --warnings=cross
| configure.in:496: error: possibly undefined macro: AM_GNU_GETTEXT_VERSION
|       If this token and others are legitimate, please use m4_pattern_allow.
|       See the Autoconf documentation.
| configure.in:497: error: possibly undefined macro: AM_GNU_GETTEXT
| autoreconf: /media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/autoconf

Quoted 6 lines

failed with exit status: 1
| ERROR: autoreconf execution failed.
NOTE: package shadow-native-4.1.4.3-r3: task do_configure: Failed
ERROR: Task 1316 (/home/steve/source/yocto/poky/meta/recipes-extended/shadow/shadow-native_4.1.4.3.bb, do_configure) failed with exit code '1'

This looks like a missing gettext dependency...

Quoted 27 lines

sqlite:
ERROR: Function failed: do_configure (see /media/data/yocto/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/sqlite-2.8.17-r7/temp/log.do_configure.20769 for further information)
ERROR: Logfile of failure stored in:
/media/data/yocto/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/sqlite-2.8.17-r7/temp/log.do_configure.20769 Log data follows:
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/omap3-multi/usr/include/sqlite.h
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/omap3-multi/usr/lib/libsqlite.so.0
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/omap3-multi/usr/lib/libsqlite.so.0.8
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/omap3-multi/usr/lib/libsqlite.so.0.8.6
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/omap3-multi/usr/lib/libsqlite.so
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/omap3-multi/usr/lib/pkgconfig/sqlite.pc
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/omap3-multi/usr/lib/pkgconfig/
| DEBUG: Removing manifest:
/media/data/yocto/tmp/sysroots/omap3-multi/usr/include/
| DEBUG: Removing manifest: /media/data/yocto/tmp/sysroots/omap3-multi/usr/lib/
| DEBUG: Removing manifest: /media/data/yocto/tmp/sysroots/omap3-multi/usr/
| DEBUG: SITE files ['endian-little', 'bit-32', 'arm-common',

Quoted 13 lines

'common-linux', 'common-glibc', 'arm-linux', 'arm-linux-gnueabi',
'common']
| ERROR: Function failed: do_configure (see /media/data/yocto/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/sqlite-2.8.17-r7/temp/log.do_configure.20769 for further information)
| main.mk is patched, no need to configure
| mv: cannot stat `/media/data/yocto/tmp/work/armv7a-vfp-neon-poky-linux-gnueabi/sqlite-2.8.17-r7/sqlite.pc':
No such file or directory
NOTE: package sqlite-2.8.17-r7: task do_configure: Failed
ERROR: Task 1673 (/home/steve/source/yocto/poky/meta-openembedded/meta-oe/recipes-support/sqlite/sqlite_2.8.17.bb, do_configure) failed with exit code '1'

I've just realised, this is from meta-oe, not OE-Core. sqlite3 doesn't seem to have this problem...

Cheers,

Richard

Richard Purdie

author

2012-04-02 03:55 UTC #5 lore raw

Quoted 44 lines · Richard Purdie

Quoted 36 lines · Steve Sakoman
Quoted 3 lines · Richard Purdie
Hmm, yes. We already have a bug open against git but I'd like to understand shadow-native and sqlite. If you have the failure logs for those two, I think it would be worth opening bugs for them.
Details for shadow-native and sqlite below:
shadow-native:
| aclocal: unknown warning category `cross'
| configure.in:496: warning: macro `AM_GNU_GETTEXT_VERSION' not found in library
| configure.in:497: warning: macro `AM_GNU_GETTEXT' not found in library
| autoreconf: configure.in: tracing
| autoreconf: configure.in: AM_GNU_GETTEXT_VERSION is used, but not
AM_GNU_GETTEXT
| autoreconf: running: libtoolize --copy --force
| libtoolize: putting auxiliary files in `.'.
| libtoolize: copying file `./ltmain.sh'
| libtoolize: Consider adding `AC_CONFIG_MACRO_DIR([m4])' to configure.in and
| libtoolize: rerunning libtoolize, to keep the correct libtool macros in-tree.
| libtoolize: Consider adding `-I m4' to ACLOCAL_AMFLAGS in Makefile.am.
| aclocal: unknown warning category `cross'
| configure.in:496: warning: macro `AM_GNU_GETTEXT_VERSION' not found in library
| configure.in:497: warning: macro `AM_GNU_GETTEXT' not found in library
| autoreconf: running:
/media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/autoconf --include=/media/data/yocto/tmp/sysroots/x86_64-linux/usr/share/aclocal-1.11 --include=/media/data/yocto/tmp/work/x86_64-linux/shadow-native-4.1.4.3-r3/shadow-4.1.4.3/aclocal-copy/ --force --warnings=cross
| configure.in:496: error: possibly undefined macro: AM_GNU_GETTEXT_VERSION
|       If this token and others are legitimate, please use m4_pattern_allow.
|       See the Autoconf documentation.
| configure.in:497: error: possibly undefined macro: AM_GNU_GETTEXT
| autoreconf: /media/data/yocto/tmp/sysroots/x86_64-linux/usr/bin/autoconf
Quoted 6 lines
failed with exit status: 1
| ERROR: autoreconf execution failed.
NOTE: package shadow-native-4.1.4.3-r3: task do_configure: Failed
ERROR: Task 1316 (/home/steve/source/yocto/poky/meta/recipes-extended/shadow/shadow-native_4.1.4.3.bb, do_configure) failed with exit code '1'

Quoted 1 line

This looks like a missing gettext dependency...

I think its more subtle than that. I think gettext-minimal-native and gettext-native are trampling over each other's files. This can probably be reproduced with:

bitbake gettext-minimal-native
bitbake gettext-native
bitbake gettext-native -c clean
bitbake shadow-native

I think the fix should be something like:

meta/recipes-core/gettext/gettext_0.18.1.1.bb +5 -1

index 4144d2b..0ed5dd9 100644
@@ -7,7 +7,7 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504"
 
 PR = "r10"
 DEPENDS = "libxml2-native gettext-native virtual/libiconv ncurses expat"
-DEPENDS_virtclass-native = "libxml2-native"
+DEPENDS_virtclass-native = "libxml2-native gettext-minimal-native"
 PROVIDES = "virtual/libintl virtual/gettext"
 PROVIDES_virtclass-native = "virtual/gettext-native"
 CONFLICTS_${PN} = "proxy-libintl"

@@ -95,4 +95,8 @@ do_install_append() {
     rm -f ${D}${libdir}/preloadable_libintl.so
 }
 
+do_install_append_virtclass-native () {
+	rm ${D}${datadir}/aclocal/*
+}
+
 BBCLASSEXTEND = "native nativesdk"

although I'm travelling at the moment and have not tested this yet.

Cheers,

Richard

Paul Eggleton

2012-04-02 09:12 UTC #6 lore raw

Quoted 25 lines · On Monday 02 April 2012 04:45:43 Richard Purdie wrote:

(quoted diff) +0 -0
b/meta/recipes-core/gettext/gettext_0.18.1.1.bb index 4144d2b..0ed5dd9 100644
meta/recipes-core/gettext/gettext_0.18.1.1.bb +0 -0
@@ -7,7 +7,7 @@ LIC_FILES_CHKSUM =
"file://COPYING;md5=d32239bcb673463ab874e80d47fae504"
PR = "r10"
DEPENDS = "libxml2-native gettext-native virtual/libiconv ncurses expat"
meta/recipes-core/gettext/gettext_0.18.1.1.bb +5 -1
-DEPENDS_virtclass-native = "libxml2-native"
+DEPENDS_virtclass-native = "libxml2-native gettext-minimal-native"
 PROVIDES = "virtual/libintl virtual/gettext"
 PROVIDES_virtclass-native = "virtual/gettext-native"
 CONFLICTS_${PN} = "proxy-libintl"
@@ -95,4 +95,8 @@ do_install_append() {
     rm -f ${D}${libdir}/preloadable_libintl.so
 }
 
+do_install_append_virtclass-native () {
+	rm ${D}${datadir}/aclocal/*
+}
+
 BBCLASSEXTEND = "native nativesdk"

Quoted 1 line

although I'm travelling at the moment and have not tested this yet.

I've tested this just now and it does fix the problem. The only thing missing is a PR bump.

Cheers,
Paul

Signature · 2 lines

Paul Eggleton
Intel Open Source Technology Centre

Richard Purdie

author

2012-04-02 12:08 UTC #7 lore raw

Quoted 26 lines · Paul Eggleton

Quoted 25 lines · On Monday 02 April 2012 04:45:43 Richard Purdie wrote:
(quoted diff) +0 -0
b/meta/recipes-core/gettext/gettext_0.18.1.1.bb index 4144d2b..0ed5dd9 100644
meta/recipes-core/gettext/gettext_0.18.1.1.bb +0 -0
@@ -7,7 +7,7 @@ LIC_FILES_CHKSUM =
"file://COPYING;md5=d32239bcb673463ab874e80d47fae504"
PR = "r10"
DEPENDS = "libxml2-native gettext-native virtual/libiconv ncurses expat"
meta/recipes-core/gettext/gettext_0.18.1.1.bb +5 -1
-DEPENDS_virtclass-native = "libxml2-native"
+DEPENDS_virtclass-native = "libxml2-native gettext-minimal-native"
 PROVIDES = "virtual/libintl virtual/gettext"
 PROVIDES_virtclass-native = "virtual/gettext-native"
 CONFLICTS_${PN} = "proxy-libintl"
@@ -95,4 +95,8 @@ do_install_append() {
     rm -f ${D}${libdir}/preloadable_libintl.so
 }
 
+do_install_append_virtclass-native () {
+	rm ${D}${datadir}/aclocal/*
+}
+
 BBCLASSEXTEND = "native nativesdk"

Quoted 4 lines

Quoted 1 line
although I'm travelling at the moment and have not tested this yet.
I've tested this just now and it does fix the problem. The only thing missing is a PR bump.

Thanks for testing. I've gone ahead and merged this into master with a PR bump added for both gettext and gettext-minimal-native to ensure sysroots are corrected.

Cheers,

Richard

Otavio Salvador

2012-04-06 19:06 UTC #8 lore raw

In reply to Richard Purdie

Quoted 4 lines · Richard Purdie

As some people have noticed, there are some rebuild issues happening due to sstate and the use of hashes in the stamp files. By this I mean the case where due to some checksum change, some task gets rerun and the task was not written to run a second time.

I had failures today at:

perl
python
wget

Signature · 3 lines

Otavio Salvador                             O.S. Systems
E-mail: [email protected]  http://www.ossystems.com.br
Mobile: +55 53 9981-7854              http://projetos.ossystems.com.br