bug in recipe version priorities across layers?

Hi,

I have a problem with bitbake choosing the wrong version of a recipe. The facts are:

- I have multiple layers, including a machine layer, meta-opendreambox, meta-openembedded and openembedded-core.
- meta-opendreambox contains wpa-supplicant_0.5.10.bb and wpa-supplicant_0.7.3.bbappend.
- openembedded-core contains wpa-supplicant_0.7.3.bb.
- There's no PREFERRED_VERSION statement for wpa-supplicant in any of the active layers. There is such a statement in another machine's layer, which shares the same TMPDIR, but even has a different PACKAGE_ARCH.
- There's no DEFAULT_PREFERENCE statement in any of the recipes.
- The machine I'm building for shall use the latest available version.

- The older wpa-supplicant is provided for a different machine.
- Both bitbake and openembedded-core are from today's master branches.

The problem: The machine I'm building for also picks up version 0.5.10, unless I eiher copy 0.7.3 from openembedded-core to my layer or delete version 0.5.10 from my layer. Deleting wpa-supplicant_0.7.3.bbappend doesn't change the behaviour.

How can copying the recipe make a difference? The BBFILE_PRIORITY for meta-opendreambox is higher than for openembedded-core, but I wouldn't expect this to have an effect on recipes with different versions. Is this expected behaviour?

Here's a log of all lines matching wpa-supplicant of "bitbake -ccleansstate wpa-supplicant" with debug enabled:

DEBUG: Cache: /a/openembedded-core/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.7.3.bb is not cached
DEBUG: Cache: /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb is not cached
DEBUG: BB /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb: handle(data)
DEBUG: LOAD /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb
DEBUG: BB /a/openembedded-core/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.7.3.bb: handle(data)
DEBUG: LOAD /a/openembedded-core/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.7.3.bb
DEBUG: CONF /a/openembedded-core/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.7.3.bb:1: including wpa-supplicant-0.7.inc
DEBUG: BB /a/openembedded-core/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant-0.7.inc: handle(data, include)
DEBUG: LOAD /a/openembedded-core/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant-0.7.inc
DEBUG: Appending .bbappend file /a/meta-openembedded/meta-oe/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.7.3.bbappend to /a/openembedded-core/meta/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.7.3.bb
DEBUG: BB /a/meta-openembedded/meta-oe/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.7.3.bbappend: handle(data, include)
DEBUG: LOAD /a/meta-openembedded/meta-oe/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.7.3.bbappend
DEBUG: providers for wpa-supplicant are: ['wpa-supplicant']
DEBUG: sorted providers for wpa-supplicant are: ['/a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb']
DEBUG: adding /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb to satisfy wpa-supplicant
DEBUG: Added dependency virtual/mipsel-oe-linux-gcc for /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb
DEBUG: Added dependency virtual/mipsel-oe-linux-compilerlibs for /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb
DEBUG: Added dependency virtual/libc for /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb
DEBUG: Added dependency openssl for /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb
DEBUG: Added runtime dependency wpa-supplicant for /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb
DEBUG: Added runtime dependency wpa-supplicant-dev for /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb
DEBUG: Added runtime recommendation wpa-supplicant for /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb
DEBUG: Added runtime recommendation wpa-supplicant-passphrase for /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb
DEBUG: providers for wpa-supplicant are: ['wpa-supplicant']
DEBUG: checking PREFERRED_PROVIDER_wpa-supplicant (value None) against ['wpa-supplicant']
DEBUG: checking PREFERRED_PROVIDER_wpa-supplicant-0.5.10 (value None) against ['wpa-supplicant']
DEBUG: checking PREFERRED_PROVIDER_wpa-supplicant-0.5.10-r0 (value None) against ['wpa-supplicant']
DEBUG: checking PREFERRED_PROVIDER_wpa-supplicant-0.7.3 (value None) against ['wpa-supplicant']
DEBUG: checking PREFERRED_PROVIDER_wpa-supplicant-0.7.3-r7 (value None) against ['wpa-supplicant']
DEBUG: sorted providers for wpa-supplicant are: ['/a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb']
DEBUG: adding '/a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb' to satisfy runtime 'wpa-supplicant'
DEBUG: providers for wpa-supplicant-dev are: ['wpa-supplicant']
DEBUG: checking PREFERRED_PROVIDER_wpa-supplicant (value None) against ['wpa-supplicant']
DEBUG: checking PREFERRED_PROVIDER_wpa-supplicant-0.5.10 (value None) against ['wpa-supplicant']
DEBUG: checking PREFERRED_PROVIDER_wpa-supplicant-0.5.10-r0 (value None) against ['wpa-supplicant']
DEBUG: checking PREFERRED_PROVIDER_wpa-supplicant-0.7.3 (value None) against ['wpa-supplicant']
DEBUG: checking PREFERRED_PROVIDER_wpa-supplicant-0.7.3-r7 (value None) against ['wpa-supplicant']
DEBUG: sorted providers for wpa-supplicant-dev are: ['/a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb']
DEBUG: adding '/a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb' to satisfy runtime 'wpa-supplicant-dev'
DEBUG: providers for wpa-supplicant-passphrase are: ['wpa-supplicant']
DEBUG: checking PREFERRED_PROVIDER_wpa-supplicant (value None) against ['wpa-supplicant']
DEBUG: checking PREFERRED_PROVIDER_wpa-supplicant-0.5.10 (value None) against ['wpa-supplicant']
DEBUG: checking PREFERRED_PROVIDER_wpa-supplicant-0.5.10-r0 (value None) against ['wpa-supplicant']
DEBUG: checking PREFERRED_PROVIDER_wpa-supplicant-0.7.3 (value None) against ['wpa-supplicant']
DEBUG: checking PREFERRED_PROVIDER_wpa-supplicant-0.7.3-r7 (value None) against ['wpa-supplicant']
DEBUG: sorted providers for wpa-supplicant-passphrase are: ['/a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb']
DEBUG: adding '/a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb' to satisfy runtime 'wpa-supplicant-passphrase'
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_unpack
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_patch
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_install
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_populate_sysroot
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_populate_sysroot_setscene
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_listtasks
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_clean
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_checkuri
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_checkuriall
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_fetch
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_fetchall
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_build
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_buildall
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_configure
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_compile
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_package_write
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_package
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_cleansstate
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_cleanall
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_rm_work
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_rm_work_all
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_recipe_sanity
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_recipe_sanity_all
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_populate_lic
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_package_setscene
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_package_write_ipk
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_package_write_ipk_setscene
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_devshell
DEBUG: Processing /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb:do_populate_lic_setscene
DEBUG: Considering 0 (/a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb, do_clean): set([1])
DEBUG: Considering 1 (/a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb, do_cleansstate): set([])
DEBUG: Stampfile /a/tmp/stamps/mipsel-oe-linux/wpa-supplicant-0.5.10-r0.do_clean not available
NOTE: Running task 1 of 2 (ID: 0, /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb, do_clean)
NOTE: package wpa-supplicant-0.5.10-r0: task do_clean: Started
NOTE: package wpa-supplicant-0.5.10-r0: task do_clean: Succeeded
DEBUG: Marking task 1 (/a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb, do_cleansstate) as buildable
DEBUG: Stampfile /a/tmp/stamps/mipsel-oe-linux/wpa-supplicant-0.5.10-r0.do_cleansstate not available
NOTE: Running task 2 of 2 (ID: 1, /a/meta-opendreambox/recipes-connectivity/wpa-supplicant/wpa-supplicant_0.5.10.bb, do_cleansstate)
NOTE: package wpa-supplicant-0.5.10-r0: task do_cleansstate: Started
NOTE: package wpa-supplicant-0.5.10-r0: task do_cleansstate: Succeeded

Regards,
Andreas

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

Afaik latest version from layer with highest priority wins. Which doesn't always mean latest version globally.

DEFAULT_PREFERENCE doesn't help e.g. to demote development version in some layer with higher priority.

But PREFERRED_VERSION does work.

Quoted 7 lines

- The older wpa-supplicant is provided for a different machine.
- Both bitbake and openembedded-core are from today's master branches.

The problem: The machine I'm building for also picks up version 0.5.10, unless I eiher copy 0.7.3 from openembedded-core to my layer or delete version 0.5.10 from my layer. Deleting wpa-supplicant_0.7.3.bbappend doesn't change the behaviour.

Quoted 7 lines · Martin Jansa

Afaik latest version from layer with highest priority wins. Which doesn't always mean latest version globally.

DEFAULT_PREFERENCE doesn't help e.g. to demote development version in some layer with higher priority.

But PREFERRED_VERSION does work.

Thanks, Martin! I didn't expect that, but it's an explanation. Is this documented somewhere? I think OE-classic behaved differently.

I noticed that "bitbake-layers show-overlayed" nicely prints the available versions in order of preference. For some packages, I don't understand the ordering though, e.g.:

python-setuptools:
meta-oe 0.6c11-ml3
meta 0.6c11-ml3
python-setuptools-native:
meta 0.6c11-ml3
meta-oe 0.6c11-ml3

(I modified bitbake-layers to include PR in its output)

The diff between both layers looks like this:

meta-oe/recipes-devtools/python/python-setuptools_0.6c11.bb renamed from meta/recipes-devtools/python/python-setuptools_0.6c11.bb +2 -1

@@ -29,3 +29,5 @@
 SRC_URI[sha256sum] = "630fea9b726320b73ee3ca6ff61732cb32675b0389be658080fe46383b87a1d3"
 
 BBCLASSEXTEND = "native"
+
+NATIVE_INSTALL_WORKS = "1"
---

The only occurence of python-setuptools(-native) is:
meta/classes/setuptools.bbclass:DEPENDS += "python-setuptools-native"

Another strange one, this time with unequal versions:

mpfr:
meta-oe 3.0.0-r4
meta 3.1.0-r0
mpfr-native:
meta 3.1.0-r0
meta-oe 3.0.0-r4

Would lowering the priority of meta-openembedded affect its ability to bbappend OE-core's recipes? Is it possible to lower its priority, or to raise the priority of OE core, without modifying conf/layer.conf in either layer?

Quoted 4 lines

Would lowering the priority of meta-openembedded affect its ability to bbappend OE-core's recipes? Is it possible to lower its priority, or to raise the priority of OE core, without modifying conf/layer.conf in either layer?

I don't remember exact rules now, but priority is strange combination of layer.conf priority and order of bblayers.conf which doesn't even behave the same for .bbclasses and .bb files IIRC, there was discussion about improving this situation but I haven't seen any patches doing it since then. FWIW .bbappend files work as expected in any layer.

please just send patch removing python-setuptools from meta-oe, NATIVE_INSTALL_WORKS isn't needed in oe-core world.. and now with python-setuptools_0.6c11.bb in oe-core it doesn't make sense to keep both :).

Quoted 8 lines

Another strange one, this time with unequal versions:

mpfr:
meta-oe 3.0.0-r4
meta 3.1.0-r0
mpfr-native:
meta 3.1.0-r0
meta-oe 3.0.0-r4

This also looks like candidate for removal or at least conversion to .bbappend if there is significant diff between meta-oe and oe-core.