Skip to contentSkip to conversation

Layer Priority with Wildcard .bbappend Files

DISCUSSION Christian Magnusson 2 participants 2 messages May 16, 2015

lore
Share & tools
lore
#0 lore raw

I also stumbled into this problem.
We are using a BSP from Phytec. It's using poky and Phytec made a simple distribution in meta-yougurt.
Now I want to customize it, and made my own meta-platform to overwrite files from the other layers. (prio 100)

# bitbake-layers show-recipes
init-ifupdown:
  meta                 1.0
# bitbake-layers show-layers
layer                 path                                      priority
==========================================================================
meta                  /home/s22463/yocto/sources/poky/meta      5
meta-yocto            /home/s22463/yocto/sources/poky/meta-yocto  5
meta-phytec           /home/s22463/yocto/sources/meta-phytec    20
meta-phyam335x        /home/s22463/yocto/sources/meta-phytec/meta-phyam335x
20
meta-yogurt           /home/s22463/yocto/sources/meta-yogurt    10
meta-oe               /home/s22463/yocto/sources/meta-openembedded/meta-oe
6
meta-networking
/home/s22463/yocto/sources/meta-openembedded/meta-networking  5
meta-python
/home/s22463/yocto/sources/meta-openembedded/meta-python  7
meta-systemd
/home/s22463/yocto/sources/meta-openembedded/meta-systemd  7
meta-qt5              /home/s22463/yocto/sources/meta-qt5       7
meta-platform         /home/s22463/yocto/sources/meta-platform  100
meta-webserver
/home/s22463/yocto/sources/meta-openembedded/meta-webserver  6
# bitbake-layers show-appends
init-ifupdown_1.0.bb:

/home/s22463/yocto/sources/meta-platform/recipes-core/init-ifupdown/init-ifu pdown_1.0.bbappend

/home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupd own_%.bbappend

Changing BBFILE_PRIORITY_platform-layer in meta-platform/conf/layer.conf doesn't affect the order either.

If I rename /home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupd own_%.bbappend -> init-ifupdown_1.0.bbappend
the priority works, and my files in meta-platform are installed last, as I want to overwrite all other files.

# mv
/home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupd
own_%.bbappend
/home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupd
own_1.0.bbappend
# bitbake-layers show-appends
init-ifupdown_1.0.bb:

/home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupd own_1.0.bbappend

/home/s22463/yocto/sources/meta-platform/recipes-core/init-ifupdown/init-ifu pdown_1.0.bbappend

The installation order works now, but I would rather avoid to patch the other layers.
Changing BBFILE_PRIORITY_platform-layer in meta-platform/conf/layer.conf now, will affect the order!, and I can install meta-plaform either before or after the meta-yougurt layer if I want.

Does anyone know the reason to the faulty sort-order?

/Christian Magnusson

On 11/24/2014 03:22 PM, Stevens, Nick wrote:
I think I've encountered a bug with how multiple bbappend files are processed
when one of the bbappends contains a filename wildcard, but I want to make sure
there's not something I'm missing before filing a bug report.

I have a BSP layer and a customization layer that are based on the OE/poky Daisy release. The output of bitbake-layers show-layers looks like:
layer path priority
=====================================================
meta poky/meta 5
meta-yocto poky/meta-yocto 5
meta-yocto-bsp poky/meta-yocto-bsp 5
...ellided...
meta-bsp meta-bsp 6
meta-custom meta-custom 8

In meta-bsp there is a bbappend for base-files named base-files_3.0.14.bbappend. The meta-bsp bbappend adds a sysctl.conf to /etc -
pretty straightforward:

     FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"
     SRC_URI += "file://sysctl.conf"
     do_install_append() {
         install -m 0644 ${WORKDIR}/sysctl.conf ${D}${sysconfdir}/
     }

Now what I want to do is add a file called base-files_%.bbappend to meta-custom. It has its own version of sysctl.conf, and the recipe looks like
this:

     FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"

Here's where things get weird. If I run bitbake -e base-files and pull out

the section for FILESEXTRAPATHS, this is what I get (note that I've stripped

out the huge absolute paths to make this easier to read):
# $FILESEXTRAPATHS [5 operations]
# set poky/meta/conf/documentation.conf:172
# [doc] "Extends the search path the OpenEmbedded build system uses
when
looking for files and patches as it processes recipes and append files."
# _prepend
meta-custom/recipes-core/base-files/base-files_%.bbappend:6
# "meta-custom/recipes-core/base-files/base-files:"
# _prepend
meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3
# "meta-bsp/recipes-core/base-files/base-files:"
# set data_smart.py:432 [finalize]
# "meta-custom/recipes-core/base-files/base-files:"
# set data_smart.py:432 [finalize]
#
"meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-core/base-f iles/base-files:"

     # computed:
     #   

"meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-core/base-f iles/base-files:"

FILESEXTRAPATHS="meta-bsp/recipes-core/base-files/base-files:meta-custom/rec ipes-core/base-files/base-files:"

For some reason meta-bsp is coming before meta-custom in FILESEXTRAPATHS, even
though meta-custom has a higher priority!

I also have some questions about how FILESEXTRAPATHS is supposed to work with append files and layer priorities.

Does a higher layer priority mean that the FILESEXTRAPATHS operation should occur first? If this is the case, then a lower priority layer prepend will appear to the left of the higher layer prepend since it runs later, and this will probably not provide the result you want.

The prepend and layer logic seems messy to me. I would like to have a way to write my custom layer and specify that what I include in my SRC_URI in each recipe is definitive and can not be overwritten. Suggestions on how to best accomplish this will be greatly appreciated.

I have been trying to accomplish this with FILESOVERRIDES, but this logic seems counterintuitive since DISTROOVERRIDES take precedence over MACHINEOVERRIDES in building the file search path during the unpack task.

It seems to me that the file search path should be built using FILESOVERRIDES from left to right: TRANSLATED_TARGET_ARCH, MACHINEOVERRIDES, and then DISTROOVERRIDES (specific to generic), but this isn't what I'm seeing. Still investigating how it's all put together...

I verified this by building an image - the sysctl.conf that ends up in the final image is the one from meta-bsp, not the one from meta-custom. But if I switch the name of base-files_%.bbappend in meta-custom to base-files_3.0.14.bbappend, this is what I get:

# $FILESEXTRAPATHS [5 operations]
# set poky/meta/conf/documentation.conf:172
# [doc] "Extends the search path the OpenEmbedded build system uses
when
looking for files and patches as it processes recipes and append files."
# _prepend
meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3
# "meta-bsp/recipes-core/base-files/base-files:"
# _prepend
meta-custom/recipes-core/base-files/base-files_3.0.14.bbappend:6
# "meta-custom/recipes-core/base-files/base-files:"
# set data_smart.py:432 [finalize]
# "meta-bsp/recipes-core/base-files/base-files:"
# set data_smart.py:432 [finalize]
#
"meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-core/base-f iles/base-files:"

     # computed:
     #   

"meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-core/base-f iles/base-files:"

FILESEXTRAPATHS="meta-custom/recipes-core/base-files/base-files:meta-bsp/rec ipes-core/base-files/base-files:"

And now the sysctl.conf file is being pulled from meta-custom.

Is there something I'm missing, like some sort of decreased priorty for bbappend files with a wildcard? Or is this a bug?

Thanks for any help!

Nick Stevens

--
_______________________________________________
yocto mailing list
[email protected]
<https://lists.yoctoproject.org/listinfo/yocto>
https://lists.yoctoproject.org/listinfo/yocto

#1 lore raw

Richard fixed this in master and fido after I opened a Yocto bug about it. I don't recall the commit offhand, but you may be able to cherry-pick it.

Quoted 332 lines · Christian Magnusson

I also stumbled into this problem.

We are using a BSP from Phytec. It’s using poky and Phytec made a simple distribution in meta-yougurt.

Now I want to customize it, and made my own meta-platform to overwrite files from the other layers. (prio 100)

# bitbake-layers show-recipes

init-ifupdown:

meta 1.0

# bitbake-layers show-layers

layer path priority

==========================================================================

meta /home/s22463/yocto/sources/poky/meta 5

meta-yocto /home/s22463/yocto/sources/poky/meta-yocto 5

meta-phytec /home/s22463/yocto/sources/meta-phytec 20

meta-phyam335x /home/s22463/yocto/sources/meta-phytec/meta-phyam335x 20

meta-yogurt /home/s22463/yocto/sources/meta-yogurt 10

meta-oe /home/s22463/yocto/sources/meta-openembedded/meta-oe 6

meta-networking /home/s22463/yocto/sources/meta-openembedded/meta-networking 5

meta-python /home/s22463/yocto/sources/meta-openembedded/meta-python 7

meta-systemd /home/s22463/yocto/sources/meta-openembedded/meta-systemd 7

meta-qt5 /home/s22463/yocto/sources/meta-qt5 7

meta-platform /home/s22463/yocto/sources/meta-platform 100

meta-webserver /home/s22463/yocto/sources/meta-openembedded/meta-webserver 6

# bitbake-layers show-appends

init-ifupdown_1.0.bb:

/home/s22463/yocto/sources/meta-platform/recipes-core/init-ifupdown/init-ifupdown_1.0.bbappend

/home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupdown_%.bbappend

Changing BBFILE_PRIORITY_platform-layer in meta-platform/conf/layer.conf doesn’t affect the order either.

If I rename

/home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupdown_%.bbappend -> init-ifupdown_1.0.bbappend

the priority works, and my files in meta-platform are installed last, as I want to overwrite all other files.

# mv /home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupdown_%.bbappend /home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupdown_1.0.bbappend
# bitbake-layers show-appends

init-ifupdown_1.0.bb:

/home/s22463/yocto/sources/meta-yogurt/recipes-core/init-ifupdown/init-ifupdown_1.0.bbappend

/home/s22463/yocto/sources/meta-platform/recipes-core/init-ifupdown/init-ifupdown_1.0.bbappend

The installation order works now, but I would rather avoid to patch the other layers.

Changing BBFILE_PRIORITY_platform-layer in meta-platform/conf/layer.conf now, will affect the order!, and I can install meta-plaform either before or after the meta-yougurt layer if I want.

Does anyone know the reason to the faulty sort-order?

/Christian Magnusson

On 11/24/2014 03:22 PM, Stevens, Nick wrote:

I think I've encountered a bug with how multiple bbappend files are processed

when one of the bbappends contains a filename wildcard, but I want to make sure

there's not something I'm missing before filing a bug report.

I have a BSP layer and a customization layer that are based on the OE/poky

Daisy release. The output of bitbake-layers show-layers looks like:

layer path priority

=====================================================

meta poky/meta 5

meta-yocto poky/meta-yocto 5

meta-yocto-bsp poky/meta-yocto-bsp 5

...ellided...

meta-bsp meta-bsp 6

meta-custom meta-custom 8

In meta-bsp there is a bbappend for base-files named

base-files_3.0.14.bbappend. The meta-bsp bbappend adds a sysctl.conf to /etc -

pretty straightforward:

     FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"

     SRC_URI += "file://sysctl.conf"

     do_install_append() {

         install -m 0644 ${WORKDIR}/sysctl.conf ${D}${sysconfdir}/

     }

Now what I want to do is add a file called base-files_%.bbappend to

meta-custom. It has its own version of sysctl.conf, and the recipe looks like

this:

     FILESEXTRAPATHS_prepend := "${THISDIR}/${PN}:"

Here's where things get weird. If I run bitbake -e base-files and pull out

the section for FILESEXTRAPATHS, this is what I get (note that I've stripped

out the huge absolute paths to make this easier to read):

# $FILESEXTRAPATHS [5 operations]

# set poky/meta/conf/documentation.conf:172

# [doc] "Extends the search path the OpenEmbedded build system uses when

looking for files and patches as it processes recipes and append files."

# _prepend meta-custom/recipes-core/base-files/base-files_%.bbappend:6

# "meta-custom/recipes-core/base-files/base-files:"

# _prepend meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3

# "meta-bsp/recipes-core/base-files/base-files:"

# set data_smart.py:432 [finalize]

# "meta-custom/recipes-core/base-files/base-files:"

# set data_smart.py:432 [finalize]

#

"meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-core/base-files/base-files:"

     # computed:

     #

"meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-core/base-files/base-files:"

FILESEXTRAPATHS="meta-bsp/recipes-core/base-files/base-files:meta-custom/recipes-core/base-files/base-files:"

For some reason meta-bsp is coming before meta-custom in FILESEXTRAPATHS, even

though meta-custom has a higher priority!

I also have some questions about how FILESEXTRAPATHS is supposed to work with append files and layer priorities.

Does a higher layer priority mean that the FILESEXTRAPATHS operation should occur first? If this is the case, then a lower priority layer prepend will appear to the left of the higher layer prepend since it runs later, and this will probably not provide the result you want.

The prepend and layer logic seems messy to me. I would like to have a way to write my custom layer and specify that what I include in my SRC_URI in each recipe is definitive and can not be overwritten. Suggestions on how to best accomplish this will be greatly appreciated.

I have been trying to accomplish this with FILESOVERRIDES, but this logic seems counterintuitive since DISTROOVERRIDES take precedence over MACHINEOVERRIDES in building the file search path during the unpack task.

It seems to me that the file search path should be built using FILESOVERRIDES from left to right: TRANSLATED_TARGET_ARCH, MACHINEOVERRIDES, and then DISTROOVERRIDES (specific to generic), but this isn't what I'm seeing. Still investigating how it's all put together...

I verified this by building an image - the sysctl.conf that ends up in the final image is the one from meta-bsp, not the one from meta-custom. But
if I switch the name of base-files_%.bbappend in meta-custom to base-files_3.0.14.bbappend, this is what I get:

# $FILESEXTRAPATHS [5 operations]

# set poky/meta/conf/documentation.conf:172

# [doc] "Extends the search path the OpenEmbedded build system uses when

looking for files and patches as it processes recipes and append files."

     #   _prepend meta-bsp/recipes-core/base-files/base-files_3.0.14.bbappend:3

     #     "meta-bsp/recipes-core/base-files/base-files:"

     #   _prepend

meta-custom/recipes-core/base-files/base-files_3.0.14.bbappend:6

# "meta-custom/recipes-core/base-files/base-files:"

# set data_smart.py:432 [finalize]

# "meta-bsp/recipes-core/base-files/base-files:"

# set data_smart.py:432 [finalize]

#

"meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-core/base-files/base-files:"

     # computed:

     #

"meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-core/base-files/base-files:"

FILESEXTRAPATHS="meta-custom/recipes-core/base-files/base-files:meta-bsp/recipes-core/base-files/base-files:"

And now the sysctl.conf file is being pulled from meta-custom.

Is there something I'm missing, like some sort of decreased priorty for

bbappend files with a wildcard? Or is this a bug?

Thanks for any help!

Nick Stevens

--

_______________________________________________

yocto mailing list

[email protected]

https://lists.yoctoproject.org/listinfo/yocto

--
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto