Skip to contentSkip to conversation

Master stability update

DISCUSSION Richard Purdie 6 participants 21 messages Jan 25, 2011 – Jan 28, 2011

lore
Share & tools
lore
#0 lore raw

We merged the features into master. I just want to highlight the status of the following issues:

* pseudo problem causing failure of meta-toolchain and corruption of certain file permissions - fixed in master

* the -live image issues some people and the autobuilder were seeing - fixed in master

* the perf issue for linux-yocto-stable - fixed in master

The following are know issues with master:

* Using rm_work and switching machines to machines of the same "multimachine" architecture breaks.

* When switching machines of the same "multimachine" architecture (e.g. emenlow to atom-pc), some sstate packages are changing checksums when at first glance they shouldn't (e.g. perl do_install). A quick scan of my sstate directory:

$ ls *core2* | sed -e 's#core2-2-.*_#core2-2_#' | sort | uniq -cd | grep -v siginfo

yields:

      2 sstate-libzypp-core2-poky-linux-0.0-git0+4494797d5b0369365b1af63921de45b197ead64f-r6-core2-2_deploy-ipk.tgz
      2 sstate-libzypp-core2-poky-linux-0.0-git0+4494797d5b0369365b1af63921de45b197ead64f-r6-core2-2_deploy-rpm.tgz
      2 sstate-libzypp-core2-poky-linux-0.0-git0+4494797d5b0369365b1af63921de45b197ead64f-r6-core2-2_package.tgz
      2 sstate-libzypp-core2-poky-linux-0.0-git0+4494797d5b0369365b1af63921de45b197ead64f-r6-core2-2_populate-sysroot.tgz
      2 sstate-netbase-core2-poky-linux-4.44-r0-core2-2_deploy-ipk.tgz
      2 sstate-netbase-core2-poky-linux-4.44-r0-core2-2_deploy-rpm.tgz
      2 sstate-netbase-core2-poky-linux-4.44-r0-core2-2_package.tgz
      2 sstate-netbase-core2-poky-linux-4.44-r0-core2-2_populate-sysroot.tgz
      2 sstate-perl-core2-poky-linux-5.12.2-r0-core2-2_deploy-ipk.tgz
      2 sstate-perl-core2-poky-linux-5.12.2-r0-core2-2_deploy-rpm.tgz
      2 sstate-perl-core2-poky-linux-5.12.2-r0-core2-2_package.tgz
      2 sstate-perl-core2-poky-linux-5.12.2-r0-core2-2_populate-sysroot.tgz
      2 sstate-rpm-core2-poky-linux-5.1.10-r8-core2-2_deploy-ipk.tgz
      2 sstate-rpm-core2-poky-linux-5.1.10-r8-core2-2_deploy-rpm.tgz
      2 sstate-rpm-core2-poky-linux-5.1.10-r8-core2-2_package.tgz
      2 sstate-rpm-core2-poky-linux-5.1.10-r8-core2-2_populate-sysroot.tgz
      2 sstate-sat-solver-core2-poky-linux-0.0-git0+aa799f7bae0ec055e0e527203635001bb7346dbc-r2-core2-2_deploy-ipk.tgz
      2 sstate-sat-solver-core2-poky-linux-0.0-git0+aa799f7bae0ec055e0e527203635001bb7346dbc-r2-core2-2_deploy-rpm.tgz
      2 sstate-sat-solver-core2-poky-linux-0.0-git0+aa799f7bae0ec055e0e527203635001bb7346dbc-r2-core2-2_package.tgz
      2 sstate-sat-solver-core2-poky-linux-0.0-git0+aa799f7bae0ec055e0e527203635001bb7346dbc-r2-core2-2_populate-sysroot.tgz
      2 sstate-zypper-core2-poky-linux-1.4.7-git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r3-core2-2_deploy-ipk.tgz
      2 sstate-zypper-core2-poky-linux-1.4.7-git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r3-core2-2_deploy-rpm.tgz
      2 sstate-zypper-core2-poky-linux-1.4.7-git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r3-core2-2_package.tgz
      2 sstate-zypper-core2-poky-linux-1.4.7-git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r3-core2-2_populate-sysroot.tgz

which isn't too bad but shows there are some issues creeping in there. These need investigating and fixing, there is a strong rpm theme going on there. Also note this was just a minimal image, sato will likely reveal other issues.

Cheers,

Richard

#1 lore raw
Quoted 9 lines · Richard Purdie

We merged the features into master. I just want to highlight the status of the following issues:

* pseudo problem causing failure of meta-toolchain and corruption of certain file permissions - fixed in master

* the -live image issues some people and the autobuilder were seeing
- fixed in master

Quoted 6 lines

* the perf issue for linux-yocto-stable - fixed in master

The following are know issues with master:

* Using rm_work and switching machines to machines of the same "multimachine" architecture breaks.

After sysroot is per machine, we may need to reserve the "image" folder even when rm_work, since for the second machine build, do_populate_sysroot and do_package will be re-run to populate files into a different sysroot folder.

Quoted 6 lines

* When switching machines of the same "multimachine" architecture (e.g.
emenlow to atom-pc), some sstate packages are changing checksums when at first glance they shouldn't (e.g. perl do_install). A quick scan of my sstate directory:

I also have a look at my local build sstate directories, there are 17 packages whose prebuilt result could not shared betweeen atom-pc and emenlow. Some of them may be not a problem since there are ${MACHINE} variables used in do_install(), some others are strange why do_install will have two different signatures between two different machines.

I will spend some time investigating it.

Thanks,
Dongxiao

Quoted 78 lines
$ ls *core2* | sed -e 's#core2-2-.*_#core2-2_#' | sort | uniq -cd |
grep -v siginfo 

yields:

      2

sstate-libzypp-core2-poky-linux-0.0-git0+4494797d5b0369365b1af63921de45b197ead64f-r6-core2-2_deploy-ipk.tgz 2 sstate-libzypp-core2-poky-linux-0.0-git0+4494797d5b0369365b1af63921de45b197ead64f-r6-core2-2_deploy-rpm.tgz 2 sstate-libzypp-core2-poky-linux-0.0-git0+4494797d5b0369365b1af63921de45b197ead64f-r6-core2-2_package.tgz 2 sstate-libzypp-core2-poky-linux-0.0-git0+4494797d5b0369365b1af63921de45b197ead64f-r6-core2-2_populate-sysroot.tgz 2 sstate-netbase-core2-poky-linux-4.44-r0-core2-2_deploy-ipk.tgz 2 sstate-netbase-core2-poky-linux-4.44-r0-core2-2_deploy-rpm.tgz 2 sstate-netbase-core2-poky-linux-4.44-r0-core2-2_package.tgz 2 sstate-netbase-core2-poky-linux-4.44-r0-core2-2_populate-sysroot.tgz
2 sstate-perl-core2-poky-linux-5.12.2-r0-core2-2_deploy-ipk.tgz 2 sstate-perl-core2-poky-linux-5.12.2-r0-core2-2_deploy-rpm.tgz 2 sstate-perl-core2-poky-linux-5.12.2-r0-core2-2_package.tgz 2 sstate-perl-core2-poky-linux-5.12.2-r0-core2-2_populate-sysroot.tgz 2 sstate-rpm-core2-poky-linux-5.1.10-r8-core2-2_deploy-ipk.tgz 2 sstate-rpm-core2-poky-linux-5.1.10-r8-core2-2_deploy-rpm.tgz 2 sstate-rpm-core2-poky-linux-5.1.10-r8-core2-2_package.tgz 2 sstate-rpm-core2-poky-linux-5.1.10-r8-core2-2_populate-sysroot.tgz 2 sstate-sat-solver-core2-poky-linux-0.0-git0+aa799f7bae0ec055e0e527203635001bb7346dbc-r2-core2-2_deploy-ipk.tgz 2 sstate-sat-solver-core2-poky-linux-0.0-git0+aa799f7bae0ec055e0e527203635001bb7346dbc-r2-core2-2_deploy-rpm.tgz 2 sstate-sat-solver-core2-poky-linux-0.0-git0+aa799f7bae0ec055e0e527203635001bb7346dbc-r2-core2-2_package.tgz 2 sstate-sat-solver-core2-poky-linux-0.0-git0+aa799f7bae0ec055e0e527203635001bb7346dbc-r2-core2-2_populate-sysroot.tgz 2 sstate-zypper-core2-poky-linux-1.4.7-git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r3-core2-2_deploy-ipk.tgz 2 sstate-zypper-core2-poky-linux-1.4.7-git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r3-core2-2_deploy-rpm.tgz 2 sstate-zypper-core2-poky-linux-1.4.7-git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r3-core2-2_package.tgz 2 sstate-zypper-core2-poky-linux-1.4.7-git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r3-core2-2_populate-sysroot.tgz

which isn't too bad but shows there are some issues creeping in there. These need investigating and fixing, there is a strong rpm theme going on there. Also note this was just a minimal image, sato will likely reveal other issues.

Cheers,

Richard

_______________________________________________
poky mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/poky

#2 lore raw
Quoted 31 lines

From: Xu, Dongxiao
Sent: Wednesday, January 26, 2011 10:46 AM

Quoted 9 lines · Richard Purdie

We merged the features into master. I just want to highlight the status of the following issues:

* pseudo problem causing failure of meta-toolchain and corruption of certain file permissions - fixed in master

* the -live image issues some people and the autobuilder were seeing
- fixed in master

Quoted 6 lines

* the perf issue for linux-yocto-stable - fixed in master

The following are know issues with master:

* Using rm_work and switching machines to machines of the same "multimachine" architecture breaks.

After sysroot is per machine, we may need to reserve the "image" folder even when rm_work, since for the second machine build, do_populate_sysroot and do_package will be re-run to populate files into a different sysroot folder.

Quoted 6 lines

* When switching machines of the same "multimachine" architecture (e.g.
emenlow to atom-pc), some sstate packages are changing checksums when at first glance they shouldn't (e.g. perl do_install). A quick scan of my sstate directory:

Quoted 8 lines

I also have a look at my local build sstate directories, there are 17 packages whose prebuilt result could not shared betweeen atom-pc and emenlow. Some of them may be not a problem since there are ${MACHINE} variables used in do_install(), some others are strange why do_install will have two different signatures between two different machines.

I will spend some time investigating it.

Note that checksum changes may not come from do_install itself, which including the hash from other tasks that do_install depends on. To capture the latter possibility, you can simply use "bitbake -S poky-image-minimal" which only generates .siginfo for all the tasks w/o actually running them. This way you can compare the difference between emenlow and atom-pc easily to see what actually change.

Thanks
Kevin

#3 lore raw
Quoted 45 lines

From: Tian, Kevin
Sent: Wednesday, January 26, 2011 10:52 AM

Quoted 13 lines

From: Xu, Dongxiao
Sent: Wednesday, January 26, 2011 10:46 AM

Quoted 9 lines · Richard Purdie

We merged the features into master. I just want to highlight the status of the following issues:

* pseudo problem causing failure of meta-toolchain and corruption of certain file permissions - fixed in master

* the -live image issues some people and the autobuilder were seeing
- fixed in master

Quoted 8 lines
Quoted 6 lines

* the perf issue for linux-yocto-stable - fixed in master

The following are know issues with master:

* Using rm_work and switching machines to machines of the same "multimachine" architecture breaks.

After sysroot is per machine, we may need to reserve the "image" folder

even

Quoted 4 lines

when rm_work, since for the second machine build, do_populate_sysroot and do_package will be re-run to populate files into a different sysroot folder.

Quoted 1 line
Quoted 8 lines
Quoted 5 lines

* When switching machines of the same "multimachine" architecture (e.g.
emenlow to atom-pc), some sstate packages are changing checksums when at first glance they shouldn't (e.g. perl do_install). A quick scan of my sstate directory:

I also have a look at my local build sstate directories, there are 17 packages whose prebuilt result could not shared betweeen atom-pc and emenlow.

Some

Quoted 6 lines

of them may be not a problem since there are ${MACHINE} variables used in do_install(), some others are strange why do_install will have two different signatures between two different machines.

I will spend some time investigating it.

Quoted 9 lines

Note that checksum changes may not come from do_install itself, which including
the hash from other tasks that do_install depends on. To capture the latter possibility, you can simply use "bitbake -S poky-image-minimal" which only generates
.siginfo for all the tasks w/o actually running them. This way you can compare the
difference between emenlow and atom-pc easily to see what actually change.

After a quick check, I guess this is not a problem. for example perl.do_configure changes checksum due to $MACHINE in dependency variables as Dongxiao said. Once perl is not reusable, at least ~10 recieps (libzypp, libtool, rpm, ...) are not reusable too because perl is in their task dependency chain.

I didn't check all the differences yet, but overall feeling is that this should be OK as Dongxiao mentioned there're several obvious MACHINE related recipes. I'll leave to Dongxiao for final confirmation. :-)

Thanks
Kevin

#4 lore raw
Quoted 55 lines · Kevin
Quoted 45 lines

From: Tian, Kevin
Sent: Wednesday, January 26, 2011 10:52 AM

Quoted 13 lines

From: Xu, Dongxiao
Sent: Wednesday, January 26, 2011 10:46 AM

Quoted 9 lines · Richard Purdie

We merged the features into master. I just want to highlight the status of the following issues:

* pseudo problem causing failure of meta-toolchain and corruption of certain file permissions - fixed in master

* the -live image issues some people and the autobuilder were seeing
- fixed in master

Quoted 8 lines
Quoted 6 lines

* the perf issue for linux-yocto-stable - fixed in master

The following are know issues with master:

* Using rm_work and switching machines to machines of the same "multimachine" architecture breaks.

After sysroot is per machine, we may need to reserve the "image" folder

even

Quoted 4 lines

when rm_work, since for the second machine build, do_populate_sysroot and do_package will be re-run to populate files into a different sysroot folder.

Quoted 1 line
Quoted 8 lines
Quoted 5 lines

* When switching machines of the same "multimachine" architecture (e.g.
emenlow to atom-pc), some sstate packages are changing checksums when at first glance they shouldn't (e.g. perl do_install). A quick scan of my sstate directory:

I also have a look at my local build sstate directories, there are 17 packages whose prebuilt result could not shared betweeen atom-pc and emenlow.

Some

Quoted 6 lines

of them may be not a problem since there are ${MACHINE} variables used in do_install(), some others are strange why do_install will have two different signatures between two different machines.

I will spend some time investigating it.

Quoted 9 lines

Note that checksum changes may not come from do_install itself, which including
the hash from other tasks that do_install depends on. To capture the latter possibility, you can simply use "bitbake -S poky-image-minimal" which only generates
.siginfo for all the tasks w/o actually running them. This way you can compare the
difference between emenlow and atom-pc easily to see what actually change.

Quoted 8 lines

After a quick check, I guess this is not a problem. for example perl.do_configure changes checksum due to $MACHINE in dependency variables as Dongxiao said. Once perl is not reusable, at least ~10 recieps (libzypp, libtool, rpm, ...) are not reusable too because perl is in their task dependency chain.

I didn't check all the differences yet, but overall feeling is that this should be OK as Dongxiao mentioned there're several obvious MACHINE related recipes. I'll leave to Dongxiao for final confirmation. :-)

It might be nice (if not a bit dangerous) to be able to say "I want perl version X.Y.Z. I don't really care about the checksum, as long as the version matches..."

In the case of perl, this is likely to do exactly what we want/need. In the case of other packages (particularly libraries) this could cause serious problems...

--Mark

Quoted 6 lines

Thanks
Kevin
_______________________________________________
poky mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/poky

#6 lore raw
In reply to Tian, Kevin
Quoted 61 lines · Kevin
Quoted 46 lines

From: Tian, Kevin
Sent: Wednesday, January 26, 2011 10:52 AM

Quoted 15 lines

From: Xu, Dongxiao
Sent: Wednesday, January 26, 2011 10:46 AM

Quoted 11 lines · Richard Purdie

We merged the features into master. I just want to highlight the status of the following issues:

* pseudo problem causing failure of meta-toolchain and corruption of certain file permissions - fixed in master

* the -live image issues some people and the autobuilder were seeing - fixed in master

* the perf issue for linux-yocto-stable - fixed in master

Quoted 7 lines
Quoted 4 lines

The following are know issues with master:

* Using rm_work and switching machines to machines of the same "multimachine" architecture breaks.

After sysroot is per machine, we may need to reserve the "image" folder

even

Quoted 10 lines

when rm_work, since for the second machine build, do_populate_sysroot and do_package will be re-run to populate files into a different sysroot folder.

Quoted 5 lines

* When switching machines of the same "multimachine" architecture (e.g. emenlow to atom-pc), some sstate packages are changing checksums when at first glance they shouldn't (e.g. perl do_install). A quick scan of my sstate directory:

Quoted 9 lines

I also have a look at my local build sstate directories, there are 17 packages whose prebuilt result could not shared betweeen atom-pc and emenlow. Some
of them may be not a problem since there are ${MACHINE} variables used in do_install(), some others are strange why do_install will have two different signatures between two different machines.

I will spend some time investigating it.

Quoted 7 lines

Note that checksum changes may not come from do_install itself, which including the hash from other tasks that do_install depends on. To capture the latter possibility, you can simply use "bitbake -S poky-image-minimal" which only generates .siginfo for all the tasks w/o actually running them. This way you can compare the difference between emenlow and atom-pc easily to see what actually change.

After a quick check, I guess this is not a problem. for example perl.do_configure changes checksum due to $MACHINE in dependency variables as Dongxiao said.
Once perl is not reusable, at least ~10 recieps (libzypp, libtool, rpm, ...) are not reusable too because perl is in their task dependency chain.

Quoted 7 lines

I didn't check all the differences yet, but overall feeling is that this should be OK as Dongxiao mentioned there're several obvious MACHINE related recipes. I'll leave to Dongxiao for final confirmation. :-)

Thanks
Kevin

Thanks Kevin for the suggestion which can quickly track down the issue.

Within my poky-image-sdk and meta-toolchain-sdk build, there are totally 17 recipes in core2-poky-linux folder whose prebuilt result could not be shared between two machine of one architecture.

They are: connman, kexec-tools, libzypp, sat-solver, rpm, perl, matchbox-keyboard, matchbox-panel-2, netbase, pulseaudio, screenshot, task-poky-sdk, tslib, xf86-input-evdev, xf86-input-keyboard, xf86-input-mouse, zypper.

I used bitbake-diffsig tool to dump the sigdata, and results are:

Connman: its do_configure depends on hal's do_populate_sysroot, while hal is a recipe of machine specific.
Pulseaudio: its do_configure depends on hal's do_populate_sysroot, while hal is a recipe of machine specific.

Kexec-tools: its do_configure depends on linux-yocto-stable's do_populate_sysroot, which is a machine specific recipe.

Perl: ${MACHINE} in do_configure.
Rpm: its do_configure depends on perl's do_populate_sysroot.
Sat-solver: its do_configure depends on rpm's do_populate_sysroot.
Libzypp: its do_configure depends on sat-solver's do_populate_sysroot.
Zypper: its do_configure depends on libzypp's do_populate_sysroot

Matchbox-panel-2: its do_configure contains "${MACHINE_FEATURES}"
Matchbox-keyboard: its do_configure depends on matchbox-panel-2's do_populate_sysroot.
Screenshot: its do_configure depends on matchbox-panel-2's do_populate_sysroot
Task-poky-sdk: depends on matchbox-panel-2

Netbase: ${MACHINE} in do_install.
Tslib: ${MACHINE} in do_install.

Xf86-input-evdev, xf86-input-keyboard, and xf86-input-mouse: xorg-xserver dependency differences, which emenlow has its own xorg-xserver layer.

Here are some dependency graph for those recipes:

hal -------> | connman
| pulseaudio

linux-yocto-stable -------> kexec-tools
perl ------> rpm ------> sat-solver ------> libzypp ------> zypper matchbox-panel-2 ------> | matchbox-keyboard
| screenshot
| task-poky-sdk
netbase
tslib
xf86-input-evdev, xf86-input-keyboard, and xf86-input-mouse: xorg-xserver layer differences.

Therefore I confirmed that the sstate prebuilt result change for the above 17 recipes are not a problem.

Thanks,
Dongxiao

#7 lore raw
Quoted 119 lines · Dongxiao
Quoted 61 lines · Kevin
Quoted 46 lines

From: Tian, Kevin
Sent: Wednesday, January 26, 2011 10:52 AM

Quoted 15 lines

From: Xu, Dongxiao
Sent: Wednesday, January 26, 2011 10:46 AM

Quoted 11 lines · Richard Purdie

We merged the features into master. I just want to highlight the status of the following issues:

* pseudo problem causing failure of meta-toolchain and corruption of certain file permissions - fixed in master

* the -live image issues some people and the autobuilder were seeing - fixed in master

* the perf issue for linux-yocto-stable - fixed in master

Quoted 7 lines
Quoted 4 lines

The following are know issues with master:

* Using rm_work and switching machines to machines of the same "multimachine" architecture breaks.

After sysroot is per machine, we may need to reserve the "image" folder

even

Quoted 10 lines

when rm_work, since for the second machine build, do_populate_sysroot and do_package will be re-run to populate files into a different sysroot folder.

Quoted 5 lines

* When switching machines of the same "multimachine" architecture (e.g. emenlow to atom-pc), some sstate packages are changing checksums when at first glance they shouldn't (e.g. perl do_install). A quick scan of my sstate directory:

Quoted 9 lines

I also have a look at my local build sstate directories, there are 17 packages whose prebuilt result could not shared betweeen atom-pc and emenlow. Some of them may be not a problem since there are ${MACHINE} variables used in do_install(), some others are strange why do_install will have two different signatures between two different machines.

I will spend some time investigating it.

Quoted 8 lines

Note that checksum changes may not come from do_install itself, which including the hash from other tasks that do_install depends on. To capture the latter possibility, you can simply use "bitbake -S poky-image-minimal" which only generates .siginfo for all the tasks w/o actually running them. This way you can compare the difference between emenlow and atom-pc easily to see what actually change.

After a quick check, I guess this is not a problem. for example perl.do_configure changes checksum due to $MACHINE in dependency variables as Dongxiao said. Once perl is not reusable, at least ~10 recieps (libzypp, libtool, rpm, ...) are not reusable too because perl is in their task dependency chain.

Quoted 7 lines

I didn't check all the differences yet, but overall feeling is that this should be OK as Dongxiao mentioned there're several obvious MACHINE related recipes. I'll leave to Dongxiao for final confirmation. :-)

Thanks
Kevin

Thanks Kevin for the suggestion which can quickly track down the issue.

Within my poky-image-sdk and meta-toolchain-sdk build, there are totally 17 recipes in core2-poky-linux folder whose prebuilt result could not be shared between two machine of one architecture.

They are: connman, kexec-tools, libzypp, sat-solver, rpm, perl, matchbox-keyboard, matchbox-panel-2, netbase, pulseaudio, screenshot, task-poky-sdk, tslib, xf86-input-evdev, xf86-input-keyboard, xf86-input-mouse, zypper.

I used bitbake-diffsig tool to dump the sigdata, and results are:

Connman: its do_configure depends on hal's do_populate_sysroot, while hal is a recipe of machine specific.
Pulseaudio: its do_configure depends on hal's do_populate_sysroot, while hal is a recipe of machine specific.

Kexec-tools: its do_configure depends on linux-yocto-stable's do_populate_sysroot, which is a machine specific recipe.

Perl: ${MACHINE} in do_configure.
Rpm: its do_configure depends on perl's do_populate_sysroot.
Sat-solver: its do_configure depends on rpm's do_populate_sysroot.
Libzypp: its do_configure depends on sat-solver's do_populate_sysroot.
Zypper: its do_configure depends on libzypp's do_populate_sysroot

Matchbox-panel-2: its do_configure contains "${MACHINE_FEATURES}"
Matchbox-keyboard: its do_configure depends on matchbox-panel-2's do_populate_sysroot.
Screenshot: its do_configure depends on matchbox-panel-2's do_populate_sysroot
Task-poky-sdk: depends on matchbox-panel-2

Netbase: ${MACHINE} in do_install.
Tslib: ${MACHINE} in do_install.

Xf86-input-evdev, xf86-input-keyboard, and xf86-input-mouse:
xorg-xserver dependency differences, which emenlow has its own xorg-xserver layer.

Here are some dependency graph for those recipes:

hal -------> | connman
| pulseaudio

Quoted 7 lines

linux-yocto-stable -------> kexec-tools perl ------> rpm ------> sat-solver ------> libzypp ------> zypper matchbox-panel-2 ------> | matchbox-keyboard
| screenshot
| task-poky-sdk netbase tslib
xf86-input-evdev, xf86-input-keyboard, and xf86-input-mouse:
xorg-xserver layer differences.

Re-draw the graph since outlook wrongly merges some lines together

hal -------> | connman.
| pulseaudio.

linux-yocto-stable -------> kexec-tools.
perl ------> rpm ------> sat-solver ------> libzypp ------> zypper. matchbox-panel-2 ------> | matchbox-keyboard.
| screenshot.
| task-poky-sdk.
netbase.
tslib.
xf86-input-evdev, xf86-input-keyboard, and xf86-input-mouse: xorg-xserver layer differences.

Thanks,
Dongxiao

Quoted 15 lines

Therefore I confirmed that the sstate prebuilt result change for the above 17 recipes are not a problem.

Thanks,
Dongxiao

_______________________________________________
poky mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/poky

#10 lore raw
In reply to Richard Purdie
Quoted 55 lines · Richard Purdie

We merged the features into master. I just want to highlight the status of the following issues:

* pseudo problem causing failure of meta-toolchain and corruption of certain file permissions - fixed in master

* the -live image issues some people and the autobuilder were seeing - fixed in master

* the perf issue for linux-yocto-stable - fixed in master

The following are know issues with master:

* Using rm_work and switching machines to machines of the same "multimachine" architecture breaks.

* When switching machines of the same "multimachine" architecture (e.g. emenlow to atom-pc), some sstate packages are changing checksums when at first glance they shouldn't (e.g. perl do_install). A quick scan of my sstate directory:

$ ls *core2* | sed -e 's#core2-2-.*_#core2-2_#' | sort | uniq -cd | grep -v siginfo

yields:

       2 sstate-libzypp-core2-poky-linux-0.0-git0+4494797d5b0369365b1af63921de45b197ead64f-r6-core2-2_deploy-ipk.tgz
       2 sstate-libzypp-core2-poky-linux-0.0-git0+4494797d5b0369365b1af63921de45b197ead64f-r6-core2-2_deploy-rpm.tgz
       2 sstate-libzypp-core2-poky-linux-0.0-git0+4494797d5b0369365b1af63921de45b197ead64f-r6-core2-2_package.tgz
       2 sstate-libzypp-core2-poky-linux-0.0-git0+4494797d5b0369365b1af63921de45b197ead64f-r6-core2-2_populate-sysroot.tgz
       2 sstate-netbase-core2-poky-linux-4.44-r0-core2-2_deploy-ipk.tgz
       2 sstate-netbase-core2-poky-linux-4.44-r0-core2-2_deploy-rpm.tgz
       2 sstate-netbase-core2-poky-linux-4.44-r0-core2-2_package.tgz
       2 sstate-netbase-core2-poky-linux-4.44-r0-core2-2_populate-sysroot.tgz
       2 sstate-perl-core2-poky-linux-5.12.2-r0-core2-2_deploy-ipk.tgz
       2 sstate-perl-core2-poky-linux-5.12.2-r0-core2-2_deploy-rpm.tgz
       2 sstate-perl-core2-poky-linux-5.12.2-r0-core2-2_package.tgz
       2 sstate-perl-core2-poky-linux-5.12.2-r0-core2-2_populate-sysroot.tgz
       2 sstate-rpm-core2-poky-linux-5.1.10-r8-core2-2_deploy-ipk.tgz
       2 sstate-rpm-core2-poky-linux-5.1.10-r8-core2-2_deploy-rpm.tgz
       2 sstate-rpm-core2-poky-linux-5.1.10-r8-core2-2_package.tgz
       2 sstate-rpm-core2-poky-linux-5.1.10-r8-core2-2_populate-sysroot.tgz
       2 sstate-sat-solver-core2-poky-linux-0.0-git0+aa799f7bae0ec055e0e527203635001bb7346dbc-r2-core2-2_deploy-ipk.tgz
       2 sstate-sat-solver-core2-poky-linux-0.0-git0+aa799f7bae0ec055e0e527203635001bb7346dbc-r2-core2-2_deploy-rpm.tgz
       2 sstate-sat-solver-core2-poky-linux-0.0-git0+aa799f7bae0ec055e0e527203635001bb7346dbc-r2-core2-2_package.tgz
       2 sstate-sat-solver-core2-poky-linux-0.0-git0+aa799f7bae0ec055e0e527203635001bb7346dbc-r2-core2-2_populate-sysroot.tgz
       2 sstate-zypper-core2-poky-linux-1.4.7-git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r3-core2-2_deploy-ipk.tgz
       2 sstate-zypper-core2-poky-linux-1.4.7-git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r3-core2-2_deploy-rpm.tgz
       2 sstate-zypper-core2-poky-linux-1.4.7-git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r3-core2-2_package.tgz
       2 sstate-zypper-core2-poky-linux-1.4.7-git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r3-core2-2_populate-sysroot.tgz

which isn't too bad but shows there are some issues creeping in there. These need investigating and fixing, there is a strong rpm theme going on there. Also note this was just a minimal image, sato will likely reveal other issues.

Quoted 9 lines

Cheers,

Richard

_______________________________________________
poky mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/poky

We are looking pretty good after the nightly has been running, a couple of issues were discovered by the Autobuilder, but we are more stable than we have been for a while.

Thanks to everyone for their hard work.

Seems that we have a couple of additional issues that need to be addressed.

1) Recent changes to the send-pull-request have caused the problem with the email now coming from poky-bounces.
* dvhart or sgarman should investigate this please

2) atom-pc connman failure, Dongxiao may have already submitted a patch

3) emenlow rootfs failure, may be related to zypper/rpm, need to verify with Mark and his patch.

Thanks
Sau!

#11 lore raw
Quoted 115 lines · Saul Wold
Quoted 95 lines · Richard Purdie

We merged the features into master. I just want to highlight the status of the following issues:

* pseudo problem causing failure of meta-toolchain and corruption of certain file permissions - fixed in master

* the -live image issues some people and the autobuilder were seeing -
fixed in master

* the perf issue for linux-yocto-stable - fixed in master

The following are know issues with master:

* Using rm_work and switching machines to machines of the same "multimachine" architecture breaks.

* When switching machines of the same "multimachine" architecture (e.g.
emenlow to atom-pc), some sstate packages are changing checksums when at first glance they shouldn't (e.g. perl do_install). A quick scan of
my sstate directory:

$ ls *core2* | sed -e 's#core2-2-.*_#core2-2_#' | sort | uniq -cd |
grep -v siginfo

yields:

       2

sstate-libzypp-core2-poky-linux-0.0-git0+4494797d5b0369365b1af63921de45b197ead64f-r6-core2-2_deploy-ipk.tgz 2 sstate-libzypp-core2-poky-linux-0.0-git0+4494797d5b0369365b1af63921de45b197ead64f-r6-core2-2_deploy-rpm.tgz 2 sstate-libzypp-core2-poky-linux-0.0-git0+4494797d5b0369365b1af63921de45b197ead64f-r6-core2-2_package.tgz 2 sstate-libzypp-core2-poky-linux-0.0-git0+4494797d5b0369365b1af63921de45b197ead64f-r6-core2-2_populate-sysroot.tgz 2 sstate-netbase-core2-poky-linux-4.44-r0-core2-2_deploy-ipk.tgz 2 sstate-netbase-core2-poky-linux-4.44-r0-core2-2_deploy-rpm.tgz 2 sstate-netbase-core2-poky-linux-4.44-r0-core2-2_package.tgz 2 sstate-netbase-core2-poky-linux-4.44-r0-core2-2_populate-sysroot.tgz
2 sstate-perl-core2-poky-linux-5.12.2-r0-core2-2_deploy-ipk.tgz 2 sstate-perl-core2-poky-linux-5.12.2-r0-core2-2_deploy-rpm.tgz 2 sstate-perl-core2-poky-linux-5.12.2-r0-core2-2_package.tgz 2 sstate-perl-core2-poky-linux-5.12.2-r0-core2-2_populate-sysroot.tgz
2 sstate-rpm-core2-poky-linux-5.1.10-r8-core2-2_deploy-ipk.tgz 2 sstate-rpm-core2-poky-linux-5.1.10-r8-core2-2_deploy-rpm.tgz 2 sstate-rpm-core2-poky-linux-5.1.10-r8-core2-2_package.tgz 2 sstate-rpm-core2-poky-linux-5.1.10-r8-core2-2_populate-sysroot.tgz 2 sstate-sat-solver-core2-poky-linux-0.0-git0+aa799f7bae0ec055e0e527203635001bb7346dbc-r2-core2-2_deploy-ipk.tgz 2 sstate-sat-solver-core2-poky-linux-0.0-git0+aa799f7bae0ec055e0e527203635001bb7346dbc-r2-core2-2_deploy-rpm.tgz 2 sstate-sat-solver-core2-poky-linux-0.0-git0+aa799f7bae0ec055e0e527203635001bb7346dbc-r2-core2-2_package.tgz 2 sstate-sat-solver-core2-poky-linux-0.0-git0+aa799f7bae0ec055e0e527203635001bb7346dbc-r2-core2-2_populate-sysroot.tgz 2 sstate-zypper-core2-poky-linux-1.4.7-git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r3-core2-2_deploy-ipk.tgz 2 sstate-zypper-core2-poky-linux-1.4.7-git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r3-core2-2_deploy-rpm.tgz 2 sstate-zypper-core2-poky-linux-1.4.7-git0+9eb0e248e06c8d20ad054be2439149d9ede37531-r3-core2-2_package.tgz 2
sstate-zypper-core2-poky-linux-1.4.7-git0+9eb0e248e06c8d20ad054be24391 49d9ede37531-r3-core2-2_populate-sysroot.tgz

which isn't too bad but shows there are some issues creeping in there.
These need investigating and fixing, there is a strong rpm theme going
on there. Also note this was just a minimal image, sato will likely reveal other issues.

Quoted 9 lines

Cheers,

Richard

_______________________________________________
poky mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/poky

We are looking pretty good after the nightly has been running, a couple of issues were discovered by the Autobuilder, but we are more stable than we have been for a while.

Thanks to everyone for their hard work.

Seems that we have a couple of additional issues that need to be addressed.

Quoted 6 lines

1) Recent changes to the send-pull-request have caused the problem with the email now coming from poky-bounces. * dvhart or sgarman
should investigate this please

2) atom-pc connman failure, Dongxiao may have already submitted a patch

This should be the same issue of doing "rm_work" when building the first machine (autobuilder inherits rm_work.bbclass). I just sent out a fix patch for review.

Thanks,
Dongxiao

Quoted 10 lines

3) emenlow rootfs failure, may be related to zypper/rpm, need to verify with Mark and his patch.

Thanks
Sau!
_______________________________________________
poky mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/poky

#8 lore raw
In reply to Xu, Dongxiao
Quoted 94 lines · Dongxiao
Quoted 61 lines · Kevin
Quoted 46 lines

From: Tian, Kevin
Sent: Wednesday, January 26, 2011 10:52 AM

Quoted 15 lines

From: Xu, Dongxiao
Sent: Wednesday, January 26, 2011 10:46 AM

Quoted 11 lines · Richard Purdie

We merged the features into master. I just want to highlight the status of the following issues:

* pseudo problem causing failure of meta-toolchain and corruption of certain file permissions - fixed in master

* the -live image issues some people and the autobuilder were seeing - fixed in master

* the perf issue for linux-yocto-stable - fixed in master

Quoted 7 lines
Quoted 4 lines

The following are know issues with master:

* Using rm_work and switching machines to machines of the same "multimachine" architecture breaks.

After sysroot is per machine, we may need to reserve the "image" folder

even

Quoted 10 lines

when rm_work, since for the second machine build, do_populate_sysroot and do_package will be re-run to populate files into a different sysroot folder.

Quoted 5 lines

* When switching machines of the same "multimachine" architecture (e.g. emenlow to atom-pc), some sstate packages are changing checksums when at first glance they shouldn't (e.g. perl do_install). A quick scan of my sstate directory:

Quoted 9 lines

I also have a look at my local build sstate directories, there are 17 packages whose prebuilt result could not shared betweeen atom-pc and emenlow. Some
of them may be not a problem since there are ${MACHINE} variables used in do_install(), some others are strange why do_install will have two different signatures between two different machines.

I will spend some time investigating it.

Quoted 7 lines

Note that checksum changes may not come from do_install itself, which including the hash from other tasks that do_install depends on. To capture the latter possibility, you can simply use "bitbake -S poky-image-minimal" which only generates .siginfo for all the tasks w/o actually running them. This way you can compare the difference between emenlow and atom-pc easily to see what actually change.

After a quick check, I guess this is not a problem. for example perl.do_configure changes checksum due to $MACHINE in dependency variables as Dongxiao said.
Once perl is not reusable, at least ~10 recieps (libzypp, libtool, rpm, ...) are not reusable too because perl is in their task dependency chain.

Quoted 7 lines

I didn't check all the differences yet, but overall feeling is that this should be OK as Dongxiao mentioned there're several obvious MACHINE related recipes. I'll leave to Dongxiao for final confirmation. :-)

Thanks
Kevin

Thanks Kevin for the suggestion which can quickly track down the issue.

Within my poky-image-sdk and meta-toolchain-sdk build, there are totally 17 recipes in core2-poky-linux folder whose prebuilt result could not be shared between two machine of one architecture.

They are: connman, kexec-tools, libzypp, sat-solver, rpm, perl, matchbox-keyboard, matchbox-panel-2, netbase, pulseaudio, screenshot, task-poky-sdk, tslib, xf86-input-evdev, xf86-input-keyboard, xf86-input-mouse, zypper.

I used bitbake-diffsig tool to dump the sigdata, and results are:

Connman: its do_configure depends on hal's do_populate_sysroot, while hal is a recipe of machine specific.
Pulseaudio: its do_configure depends on hal's do_populate_sysroot, while hal is a recipe of machine specific.

Kexec-tools: its do_configure depends on linux-yocto-stable's do_populate_sysroot, which is a machine specific recipe.

Perl: ${MACHINE} in do_configure.
Rpm: its do_configure depends on perl's do_populate_sysroot.
Sat-solver: its do_configure depends on rpm's do_populate_sysroot.
Libzypp: its do_configure depends on sat-solver's do_populate_sysroot.
Zypper: its do_configure depends on libzypp's do_populate_sysroot

Matchbox-panel-2: its do_configure contains "${MACHINE_FEATURES}"
Matchbox-keyboard: its do_configure depends on matchbox-panel-2's do_populate_sysroot.
Screenshot: its do_configure depends on matchbox-panel-2's do_populate_sysroot
Task-poky-sdk: depends on matchbox-panel-2

Quoted 5 lines

Netbase: ${MACHINE} in do_install.
Tslib: ${MACHINE} in do_install.

Xf86-input-evdev, xf86-input-keyboard, and xf86-input-mouse: xorg-xserver dependency differences, which emenlow has its own xorg-xserver layer.

So for each of these we need to decide if rebuilding is a valid or invalid thing to do.

The one that jumps out at me is perl. Why does it need MACHINE in do_configure and is it really MACHINE dependent? If you look at the do_configure code, its a rather strange check against the "native" machine which we don't use/support. I'd argue that machine is error prone, problematic and replaced by our nativesdk code so I'm in favour of removing that check.

Some of the other items above are trickier. In some cases we should perhaps be marking them as machine specific, e.g. mb-panel-2 ? and maybe also whitelisting some dependencies if we're sure those items don't change if the dependency changes.

The nice thing is now that we can make something like mb-panel-2 machine specific and not have the whole world collapse as it would have done previously so we can start to benefit from this change! :)

Cheers,

Richard

#9 lore raw
Quoted 125 lines

From: Richard Purdie [mailto:[email protected]]
Sent: Wednesday, January 26, 2011 6:01 PM

Quoted 67 lines · Dongxiao
Quoted 66 lines · Kevin
Quoted 46 lines

From: Tian, Kevin
Sent: Wednesday, January 26, 2011 10:52 AM

Quoted 15 lines

From: Xu, Dongxiao
Sent: Wednesday, January 26, 2011 10:46 AM

Quoted 11 lines · Richard Purdie

We merged the features into master. I just want to highlight the status of the following issues:

* pseudo problem causing failure of meta-toolchain and corruption of certain file permissions - fixed in master

* the -live image issues some people and the autobuilder were seeing - fixed in master

* the perf issue for linux-yocto-stable - fixed in master

Quoted 7 lines
Quoted 4 lines

The following are know issues with master:

* Using rm_work and switching machines to machines of the same "multimachine" architecture breaks.

After sysroot is per machine, we may need to reserve the "image" folder

even

Quoted 10 lines

when rm_work, since for the second machine build, do_populate_sysroot and do_package will be re-run to populate files into a different sysroot folder.

Quoted 5 lines

* When switching machines of the same "multimachine" architecture (e.g. emenlow to atom-pc), some sstate packages are changing checksums when at first glance they shouldn't (e.g. perl do_install). A quick scan of my sstate directory:

Quoted 9 lines

I also have a look at my local build sstate directories, there are 17 packages whose prebuilt result could not shared betweeen atom-pc and emenlow. Some
of them may be not a problem since there are ${MACHINE} variables used in do_install(), some others are strange why do_install will have two different signatures between two different machines.

I will spend some time investigating it.

Quoted 7 lines

Note that checksum changes may not come from do_install itself, which including the hash from other tasks that do_install depends on. To capture the latter possibility, you can simply use "bitbake -S poky-image-minimal" which only generates .siginfo for all the tasks w/o actually running them. This way you can compare the difference between emenlow and atom-pc easily to see what actually change.

After a quick check, I guess this is not a problem. for example perl.do_configure changes checksum due to $MACHINE in dependency variables as Dongxiao said.
Once perl is not reusable, at least ~10 recieps (libzypp, libtool, rpm, ...) are not reusable too because perl is in their task dependency chain.

I didn't check all the differences yet, but overall feeling is that this should be OK as Dongxiao mentioned there're several obvious MACHINE related recipes. I'll leave to Dongxiao for final confirmation. :-)

Quoted 6 lines
Quoted 2 lines

Thanks
Kevin

Thanks Kevin for the suggestion which can quickly track down the issue.

Within my poky-image-sdk and meta-toolchain-sdk build, there are totally 17

recipes in core2-poky-linux folder whose prebuilt result could not be shared between two machine of one architecture.

Quoted 2 lines

They are: connman, kexec-tools, libzypp, sat-solver, rpm, perl,

matchbox-keyboard, matchbox-panel-2, netbase, pulseaudio, screenshot, task-poky-sdk, tslib, xf86-input-evdev, xf86-input-keyboard, xf86-input-mouse, zypper.

Quoted 4 lines

I used bitbake-diffsig tool to dump the sigdata, and results are:

Connman: its do_configure depends on hal's do_populate_sysroot, while hal

is a recipe of machine specific.

Quoted 1 line

Pulseaudio: its do_configure depends on hal's do_populate_sysroot, while hal

is a recipe of machine specific.

Quoted 2 lines

Kexec-tools: its do_configure depends on linux-yocto-stable's

do_populate_sysroot, which is a machine specific recipe.

Quoted 9 lines

Perl: ${MACHINE} in do_configure.
Rpm: its do_configure depends on perl's do_populate_sysroot.
Sat-solver: its do_configure depends on rpm's do_populate_sysroot.
Libzypp: its do_configure depends on sat-solver's do_populate_sysroot.
Zypper: its do_configure depends on libzypp's do_populate_sysroot

Matchbox-panel-2: its do_configure contains "${MACHINE_FEATURES}"
Matchbox-keyboard: its do_configure depends on matchbox-panel-2's

do_populate_sysroot.

Quoted 1 line

Screenshot: its do_configure depends on matchbox-panel-2's

do_populate_sysroot

Quoted 6 lines

Task-poky-sdk: depends on matchbox-panel-2

Netbase: ${MACHINE} in do_install.
Tslib: ${MACHINE} in do_install.

Xf86-input-evdev, xf86-input-keyboard, and xf86-input-mouse: xorg-xserver

dependency differences, which emenlow has its own xorg-xserver layer.

Quoted 1 line

So for each of these we need to decide if rebuilding is a valid or invalid thing to do.

The one that jumps out at me is perl. Why does it need MACHINE in do_configure and is it really MACHINE dependent? If you look at the do_configure code, its a rather strange check against the "native" machine which we don't use/support. I'd argue that machine is error prone, problematic and replaced by our nativesdk code so I'm in favour of removing that check.

Quoted 9 lines

Some of the other items above are trickier. In some cases we should perhaps be marking them as machine specific, e.g. mb-panel-2 ? and maybe also whitelisting some dependencies if we're sure those items don't change if the dependency changes.

The nice thing is now that we can make something like mb-panel-2 machine specific and not have the whole world collapse as it would have done previously so we can start to benefit from this change! :)

Yes, this at least proves that overall now we're in a good state with recent changes. :-)

Your suggestions are good, and we'll continue look into them to reduce the machine specific bits as possible.

Thanks
Kevin

#5 lore raw
In reply to Mark Hatle
Quoted 11 lines · Mark Hatle
Quoted 9 lines · Kevin
Quoted 1 line

From: Tian, Kevin

After a quick check, I guess this is not a problem. for example perl.do_configure changes checksum due to $MACHINE in dependency variables as Dongxiao said. Once perl is not reusable, at least ~10 recieps (libzypp, libtool, rpm, ...) are not reusable too because perl is in their task dependency chain.

I didn't check all the differences yet, but overall feeling is that this should be OK as Dongxiao mentioned there're several obvious MACHINE related recipes. I'll leave to Dongxiao for final confirmation. :-)

Quoted 5 lines

It might be nice (if not a bit dangerous) to be able to say "I want perl version X.Y.Z. I don't really care about the checksum, as long as the version matches..."

In the case of perl, this is likely to do exactly what we want/need. In the case of other packages (particularly libraries) this could cause serious problems...

Adding "perl" to BB_HASHTASK_WHITELIST would do exactly what you describe so its at least possible although it doesn't cover a version specific match which would probably be better as part of DEPENDS handling.

Stopping it being machine specific is the first thing that needs fixing though (I have a patch I'm testing for that one).

Cheers,

Richard

#12 lore raw
In reply to Richard Purdie

Op 25 jan 2011, om 23:07 heeft Richard Purdie het volgende geschreven:

Quoted 8 lines

The following are know issues with master:

* Using rm_work and switching machines to machines of the same "multimachine" architecture breaks.

* When switching machines of the same "multimachine" architecture (e.g. emenlow to atom-pc), some sstate packages are changing checksums when at first glance they shouldn't (e.g. perl do_install). A

After the fixes for those 2 went in builds from scratch fail with:

ERROR: Logfile of failure stored in: /OE/tentacle/build/tmp-angstrom_2010_x/work/x86_64-linux/gmp-native-5.0.1-r0/temp/log.do_configure.31887 Log data follows:

| automake (GNU automake) 1.9.6
| Written by Tom Tromey <[email protected]>.
|
| Copyright 2005 Free Software Foundation, Inc.
| This is free software; see the source for copying conditions.  There is NO
| warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
| AUTOV is 1.9
| NOTE: Executing autoreconf --verbose --install --force --exclude=autopoint -I /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/share/aclocal
| /usr/bin/autoreconf2.50: unrecognized option `--exclude=autopoint'
| Try `/usr/bin/autoreconf2.50 --help' for more information.
| FATAL: autoreconf execution failed.
| Function 'do_configure' failed (see /OE/tentacle/build/tmp-angstrom_2010_x/work/x86_64-linux/gmp-native-5.0.1-r0/temp/log.do_configure.31887 for further information)
| ERROR: Function 'do_configure' failed (see /OE/tentacle/build/tmp-angstrom_2010_x/work/x86_64-linux/gmp-native-5.0.1-r0/temp/log.do_configure.31887 for further information)

It wants to use the autoreconf from my host, which doesn't have the `--exclude=autopoint' option since that's something OE patches in. After building autoconf native pseudo is gone!

/OE/tentacle/sources/openembedded/scripts/bitbake: line 38: /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/pseudo: No such file or directory

After building pseudo-native again:

NOTE: package pseudo-native-0.0+git0+bcb42d80c0817da5479ab9c4f2cd8c4727e98ef8-r17: task do_rm_work_all: Started
NOTE: package pseudo-native-0.0+git0+bcb42d80c0817da5479ab9c4f2cd8c4727e98ef8-r17: task do_rm_work_all: Succeeded
NOTE: Tasks Summary: Attempted 785 tasks of which 448 didn't need to be rerun and 0 failed.

koen@dominion:/OE/tentacle$ MACHINE=beagleboard bitbake console-image
/OE/tentacle/sources/openembedded/scripts/bitbake: line 38: /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/pseudo: No such file or directory

weird

regards,

Koen

#13 lore raw
Quoted 32 lines · Koen Kooi

Op 25 jan 2011, om 23:07 heeft Richard Purdie het volgende geschreven:

Quoted 8 lines

The following are know issues with master:

* Using rm_work and switching machines to machines of the same "multimachine" architecture breaks.

* When switching machines of the same "multimachine" architecture (e.g. emenlow to atom-pc), some sstate packages are changing checksums when at first glance they shouldn't (e.g. perl do_install). A

After the fixes for those 2 went in builds from scratch fail with:

ERROR: Logfile of failure stored in: /OE/tentacle/build/tmp-angstrom_2010_x/work/x86_64-linux/gmp-native-5.0.1-r0/temp/log.do_configure.31887 Log data follows:

| automake (GNU automake) 1.9.6
| Written by Tom Tromey <[email protected]>.
|
| Copyright 2005 Free Software Foundation, Inc.
| This is free software; see the source for copying conditions.  There is NO
| warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
| AUTOV is 1.9
| NOTE: Executing autoreconf --verbose --install --force --exclude=autopoint -I /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/share/aclocal
| /usr/bin/autoreconf2.50: unrecognized option `--exclude=autopoint'
| Try `/usr/bin/autoreconf2.50 --help' for more information.
| FATAL: autoreconf execution failed.
| Function 'do_configure' failed (see /OE/tentacle/build/tmp-angstrom_2010_x/work/x86_64-linux/gmp-native-5.0.1-r0/temp/log.do_configure.31887 for further information)
| ERROR: Function 'do_configure' failed (see /OE/tentacle/build/tmp-angstrom_2010_x/work/x86_64-linux/gmp-native-5.0.1-r0/temp/log.do_configure.31887 for further information)

It wants to use the autoreconf from my host, which doesn't have the `--exclude=autopoint' option since that's something OE patches in. After building autoconf native pseudo is gone!

/OE/tentacle/sources/openembedded/scripts/bitbake: line 38: /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/pseudo: No such file or directory

Quoted 7 lines

After building pseudo-native again:

NOTE: package pseudo-native-0.0+git0+bcb42d80c0817da5479ab9c4f2cd8c4727e98ef8-r17: task do_rm_work_all: Started
NOTE: package pseudo-native-0.0+git0+bcb42d80c0817da5479ab9c4f2cd8c4727e98ef8-r17: task do_rm_work_all: Succeeded
NOTE: Tasks Summary: Attempted 785 tasks of which 448 didn't need to be rerun and 0 failed.

koen@dominion:/OE/tentacle$ MACHINE=beagleboard bitbake console-image
/OE/tentacle/sources/openembedded/scripts/bitbake: line 38: /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/pseudo: No such file or directory

It seems that autoconf.bbclass change is having a weird ripple effect with the sstate packages. Can you clean sstate and then retry, just to try and isolate the issues?

I agree having a simple change to autotools.bbclass ripple through sstate like that isn't desirable btw, I just want to try and fix one issue at a time.

Cheers,

Richard

#14 lore raw

Op 27 jan 2011, om 16:06 heeft Richard Purdie het volgende geschreven:

Quoted 41 lines · Koen Kooi
Quoted 32 lines

Op 25 jan 2011, om 23:07 heeft Richard Purdie het volgende geschreven:

Quoted 8 lines

The following are know issues with master:

* Using rm_work and switching machines to machines of the same "multimachine" architecture breaks.

* When switching machines of the same "multimachine" architecture (e.g. emenlow to atom-pc), some sstate packages are changing checksums when at first glance they shouldn't (e.g. perl do_install). A

After the fixes for those 2 went in builds from scratch fail with:

ERROR: Logfile of failure stored in: /OE/tentacle/build/tmp-angstrom_2010_x/work/x86_64-linux/gmp-native-5.0.1-r0/temp/log.do_configure.31887 Log data follows:

| automake (GNU automake) 1.9.6
| Written by Tom Tromey <[email protected]>.
|
| Copyright 2005 Free Software Foundation, Inc.
| This is free software; see the source for copying conditions.  There is NO
| warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
| AUTOV is 1.9
| NOTE: Executing autoreconf --verbose --install --force --exclude=autopoint -I /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/share/aclocal
| /usr/bin/autoreconf2.50: unrecognized option `--exclude=autopoint'
| Try `/usr/bin/autoreconf2.50 --help' for more information.
| FATAL: autoreconf execution failed.
| Function 'do_configure' failed (see /OE/tentacle/build/tmp-angstrom_2010_x/work/x86_64-linux/gmp-native-5.0.1-r0/temp/log.do_configure.31887 for further information)
| ERROR: Function 'do_configure' failed (see /OE/tentacle/build/tmp-angstrom_2010_x/work/x86_64-linux/gmp-native-5.0.1-r0/temp/log.do_configure.31887 for further information)

It wants to use the autoreconf from my host, which doesn't have the `--exclude=autopoint' option since that's something OE patches in. After building autoconf native pseudo is gone!

/OE/tentacle/sources/openembedded/scripts/bitbake: line 38: /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/pseudo: No such file or directory

Quoted 7 lines

After building pseudo-native again:

NOTE: package pseudo-native-0.0+git0+bcb42d80c0817da5479ab9c4f2cd8c4727e98ef8-r17: task do_rm_work_all: Started
NOTE: package pseudo-native-0.0+git0+bcb42d80c0817da5479ab9c4f2cd8c4727e98ef8-r17: task do_rm_work_all: Succeeded
NOTE: Tasks Summary: Attempted 785 tasks of which 448 didn't need to be rerun and 0 failed.

koen@dominion:/OE/tentacle$ MACHINE=beagleboard bitbake console-image
/OE/tentacle/sources/openembedded/scripts/bitbake: line 38: /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/pseudo: No such file or directory
Quoted 7 lines

It seems that autoconf.bbclass change is having a weird ripple effect with the sstate packages. Can you clean sstate and then retry, just to try and isolate the issues?

I agree having a simple change to autotools.bbclass ripple through sstate like that isn't desirable btw, I just want to try and fix one issue at a time.

I removed tmp, sstate-cache and pseudodone, then did 'bitbake console-image', which failed at the same point. I looked into '/OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/', which had a lot files in it, but no autoconf, so I did 'bitbake autoconf-native', that (re)ran a lot of rm_work tasks.

koen@dominion:/OE/tentacle$ ls /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/
tclConfig.sh*

so it removed pretty much everything from the sysroot!

#15 lore raw

Op 27 jan 2011, om 16:35 heeft Koen Kooi het volgende geschreven:

Quoted 52 lines

Op 27 jan 2011, om 16:06 heeft Richard Purdie het volgende geschreven:

Quoted 41 lines · Koen Kooi
Quoted 32 lines

Op 25 jan 2011, om 23:07 heeft Richard Purdie het volgende geschreven:

Quoted 8 lines

The following are know issues with master:

* Using rm_work and switching machines to machines of the same "multimachine" architecture breaks.

* When switching machines of the same "multimachine" architecture (e.g. emenlow to atom-pc), some sstate packages are changing checksums when at first glance they shouldn't (e.g. perl do_install). A

After the fixes for those 2 went in builds from scratch fail with:

ERROR: Logfile of failure stored in: /OE/tentacle/build/tmp-angstrom_2010_x/work/x86_64-linux/gmp-native-5.0.1-r0/temp/log.do_configure.31887 Log data follows:

| automake (GNU automake) 1.9.6
| Written by Tom Tromey <[email protected]>.
|
| Copyright 2005 Free Software Foundation, Inc.
| This is free software; see the source for copying conditions.  There is NO
| warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
| AUTOV is 1.9
| NOTE: Executing autoreconf --verbose --install --force --exclude=autopoint -I /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/share/aclocal
| /usr/bin/autoreconf2.50: unrecognized option `--exclude=autopoint'
| Try `/usr/bin/autoreconf2.50 --help' for more information.
| FATAL: autoreconf execution failed.
| Function 'do_configure' failed (see /OE/tentacle/build/tmp-angstrom_2010_x/work/x86_64-linux/gmp-native-5.0.1-r0/temp/log.do_configure.31887 for further information)
| ERROR: Function 'do_configure' failed (see /OE/tentacle/build/tmp-angstrom_2010_x/work/x86_64-linux/gmp-native-5.0.1-r0/temp/log.do_configure.31887 for further information)

It wants to use the autoreconf from my host, which doesn't have the `--exclude=autopoint' option since that's something OE patches in. After building autoconf native pseudo is gone!

/OE/tentacle/sources/openembedded/scripts/bitbake: line 38: /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/pseudo: No such file or directory

Quoted 7 lines

After building pseudo-native again:

NOTE: package pseudo-native-0.0+git0+bcb42d80c0817da5479ab9c4f2cd8c4727e98ef8-r17: task do_rm_work_all: Started
NOTE: package pseudo-native-0.0+git0+bcb42d80c0817da5479ab9c4f2cd8c4727e98ef8-r17: task do_rm_work_all: Succeeded
NOTE: Tasks Summary: Attempted 785 tasks of which 448 didn't need to be rerun and 0 failed.

koen@dominion:/OE/tentacle$ MACHINE=beagleboard bitbake console-image
/OE/tentacle/sources/openembedded/scripts/bitbake: line 38: /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/pseudo: No such file or directory
Quoted 7 lines

It seems that autoconf.bbclass change is having a weird ripple effect with the sstate packages. Can you clean sstate and then retry, just to try and isolate the issues?

I agree having a simple change to autotools.bbclass ripple through sstate like that isn't desirable btw, I just want to try and fix one issue at a time.

Quoted 7 lines

I removed tmp, sstate-cache and pseudodone, then did 'bitbake console-image', which failed at the same point. I looked into '/OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/', which had a lot files in it, but no autoconf, so I did 'bitbake autoconf-native', that (re)ran a lot of rm_work tasks.

koen@dominion:/OE/tentacle$ ls /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/
tclConfig.sh*

so it removed pretty much everything from the sysroot!

RP said the following during a chat:

Its do_distribute_sources
That has triggered a new fetch call
and fetch triggered setscene
and setscene cleaned out the populate_staging

So we know what's going on, now figure out a fix :)

#16 lore raw
In reply to Koen Kooi
Quoted 17 lines · Koen Kooi

Op 27 jan 2011, om 16:06 heeft Richard Purdie het volgende geschreven:

Quoted 9 lines · Koen Kooi
Quoted 1 line

Op 25 jan 2011, om 23:07 heeft Richard Purdie het volgende geschreven:

It seems that autoconf.bbclass change is having a weird ripple effect with the sstate packages. Can you clean sstate and then retry, just to try and isolate the issues?

I agree having a simple change to autotools.bbclass ripple through sstate like that isn't desirable btw, I just want to try and fix one issue at a time.

I removed tmp, sstate-cache and pseudodone, then did 'bitbake
console-image', which failed at the same point. I looked into
'/OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/', which had a lot files in it, but no autoconf, so I did 'bitbake autoconf-native', that (re)ran a lot of rm_work tasks.

Quoted 4 lines
koen@dominion:/OE/tentacle$ ls /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/
tclConfig.sh*

so it removed pretty much everything from the sysroot!

Just to update on this, Koen is using distribute_sources which adds a do_distribute_sources task. This is why things are working on master but not for Koen.

That task runs later and causes do_setscene to rerun which cleans out the sysroot rather helpfully assuming that all tasks for that recipe will rerun which isn't true.

This branch has some proposed changes to make this work better:

http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rpurdie/misc2

and Koen is testing them to see how they work out.

Cheers,

Richard

#17 lore raw

Op 27 jan 2011, om 22:45 heeft Richard Purdie het volgende geschreven:

Quoted 31 lines · Koen Kooi
Quoted 17 lines

Op 27 jan 2011, om 16:06 heeft Richard Purdie het volgende geschreven:

Quoted 9 lines · Koen Kooi
Quoted 1 line

Op 25 jan 2011, om 23:07 heeft Richard Purdie het volgende geschreven:

It seems that autoconf.bbclass change is having a weird ripple effect with the sstate packages. Can you clean sstate and then retry, just to try and isolate the issues?

I agree having a simple change to autotools.bbclass ripple through sstate like that isn't desirable btw, I just want to try and fix one issue at a time.

I removed tmp, sstate-cache and pseudodone, then did 'bitbake
console-image', which failed at the same point. I looked into
'/OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/', which had a lot files in it, but no autoconf, so I did 'bitbake autoconf-native', that (re)ran a lot of rm_work tasks.

Quoted 4 lines
koen@dominion:/OE/tentacle$ ls /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/
tclConfig.sh*

so it removed pretty much everything from the sysroot!

Just to update on this, Koen is using distribute_sources which adds a do_distribute_sources task. This is why things are working on master but not for Koen.

That task runs later and causes do_setscene to rerun which cleans out the sysroot rather helpfully assuming that all tasks for that recipe will rerun which isn't true.

Quoted 5 lines

This branch has some proposed changes to make this work better:

http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rpurdie/misc2

and Koen is testing them to see how they work out.

It gets a lot further now, I'll check the end result tomorrow after I wake up.

regards,

Koen

#18 lore raw
In reply to Richard Purdie

Op 27 jan 2011, om 22:45 heeft Richard Purdie het volgende geschreven:

Quoted 31 lines · Koen Kooi
Quoted 17 lines

Op 27 jan 2011, om 16:06 heeft Richard Purdie het volgende geschreven:

Quoted 9 lines · Koen Kooi
Quoted 1 line

Op 25 jan 2011, om 23:07 heeft Richard Purdie het volgende geschreven:

It seems that autoconf.bbclass change is having a weird ripple effect with the sstate packages. Can you clean sstate and then retry, just to try and isolate the issues?

I agree having a simple change to autotools.bbclass ripple through sstate like that isn't desirable btw, I just want to try and fix one issue at a time.

I removed tmp, sstate-cache and pseudodone, then did 'bitbake
console-image', which failed at the same point. I looked into
'/OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/', which had a lot files in it, but no autoconf, so I did 'bitbake autoconf-native', that (re)ran a lot of rm_work tasks.

Quoted 4 lines
koen@dominion:/OE/tentacle$ ls /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/
tclConfig.sh*

so it removed pretty much everything from the sysroot!

Just to update on this, Koen is using distribute_sources which adds a do_distribute_sources task. This is why things are working on master but not for Koen.

That task runs later and causes do_setscene to rerun which cleans out the sysroot rather helpfully assuming that all tasks for that recipe will rerun which isn't true.

Quoted 5 lines

This branch has some proposed changes to make this work better:

http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rpurdie/misc2

and Koen is testing them to see how they work out.

That fixes both the setscene and machine switching issues. Everything works as expected again!

#19 lore raw
Quoted 17 lines · Koen Kooi

Op 27 jan 2011, om 22:45 heeft Richard Purdie het volgende geschreven:

Quoted 15 lines · Koen Kooi
Quoted 5 lines

Op 27 jan 2011, om 16:06 heeft Richard Purdie het volgende geschreven:

koen@dominion:/OE/tentacle$ ls /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/
tclConfig.sh*

so it removed pretty much everything from the sysroot!

Just to update on this, Koen is using distribute_sources which adds a do_distribute_sources task. This is why things are working on master but not for Koen.

That task runs later and causes do_setscene to rerun which cleans out the sysroot rather helpfully assuming that all tasks for that recipe will rerun which isn't true.

Quoted 7 lines
Quoted 5 lines

This branch has some proposed changes to make this work better:

http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rpurdie/misc2

and Koen is testing them to see how they work out.

That fixes both the setscene and machine switching issues. Everything works as expected again!

Those fixes have been merged into master now.

Cheers,

Richard

#20 lore raw

Op 28 jan 2011, om 17:59 heeft Richard Purdie het volgende geschreven:

Quoted 20 lines · Koen Kooi
Quoted 19 lines

Op 27 jan 2011, om 22:45 heeft Richard Purdie het volgende geschreven:

Quoted 17 lines · Koen Kooi
Quoted 5 lines

Op 27 jan 2011, om 16:06 heeft Richard Purdie het volgende geschreven:

koen@dominion:/OE/tentacle$ ls /OE/tentacle/build/tmp-angstrom_2010_x/sysroots/x86_64-linux/usr/bin/
tclConfig.sh*

so it removed pretty much everything from the sysroot!

Just to update on this, Koen is using distribute_sources which adds a do_distribute_sources task. This is why things are working on master but not for Koen.

That task runs later and causes do_setscene to rerun which cleans out the sysroot rather helpfully assuming that all tasks for that recipe will rerun which isn't true.

This branch has some proposed changes to make this work better:

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=rpurdie/misc2

and Koen is testing them to see how they work out.

That fixes both the setscene and machine switching issues. Everything works as expected again!

Those fixes have been merged into master now.

The problem isn't entirely gone yet. What I did

MACHINE=beagleboard bitbake console-image (worked) MACHINE=usrp-e1xx bitbake console-image (worked) MACHINE=hawkboard bitbake console-image (broke)

updated metadata

MACHINE=omap4430-panda bitbake console-image (broke)

The first breakage was a license checksum failure in libusb-compat, which I have overlayed locally, I changed that into a bbappend and rebuilt, things went fine. After switching to pandaboard a lot of things broke (pango, atk, kernel, etc), most of them in package_write. It turns out that the license failure was caused by do_unpack getting skipped, so the it couldn't find the LICENSE and COPYING files to checksum.

It seems that during do_configure the stamps don't get reset complete, so no unpack/patch is run. Due to the way the classes are structured, running in an empty dir doesn't cause problems till we encounter custom packaging rules. Does that make sense?

The kernel failure looks like this:

ERROR: Logfile of failure stored in: /OE/tentacle/build/tmp-angstrom_2010_x/work/beagleboard-angstrom-linux-gnueabi/linux-omap-psp-2.6.32-r99+gitr5fc29e7b2a76a64a739f857858ef0b98294aa155/temp/log.do_package_write_ipk.18429 Log data follows:
| Lockfile destination directory '/OE/tentacle/build/tmp-angstrom_2010_x/work/beagleboard-angstrom-linux-gnueabi/linux-omap-psp-2.6.32-r99+gitr5fc29e7b2a76a64a739f857858ef0b98294aa155/packages-split' does not exist | ERROR: Lockfile destination directory '/OE/tentacle/build/tmp-angstrom_2010_x/work/beagleboard-angstrom-linux-gnueabi/linux-omap-psp-2.6.32-r99+gitr5fc29e7b2a76a64a739f857858ef0b98294aa155/packages-split' does not exist

regards,

Koen