Skip to contentSkip to conversation

common-tasks.rst: remove SRC_URI:append from examples

PATCH Superseded Mikko Rapeli 3 participants 10 messages Sep 12, 2022 – Sep 20, 2022

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/7 common-tasks.rst: remove SRC_URI:append from examples

Using SRC_URI:append without recipe, machine or architecture specific limitations makes the :append'ed text unremovable and thus users and custom layers can not change the variable anymore. This makes it hard to e.g. override SRC_URI completely in a bbappend to update the full recipe to a newer version. Thus common, reusable layers which users are meant to re-use and customize should not use SRC_URI:append but SRC_URI += instead.

---
 documentation/dev-manual/common-tasks.rst | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)
documentation/dev-manual/common-tasks.rst +5 -4
index b08a55331d..3112f9b893 100644
@@ -2577,7 +2577,7 @@ chapter of the BitBake User Manual.
 
       S = "${WORKDIR}/postfix-${PV}"
       CFLAGS += "-DNO_ASM"
-      SRC_URI:append = " file://fixup.patch"
+      CFLAGS:append = " --enable-important-feature"
 
 -  *Functions:* Functions provide a series of actions to be performed.
    You usually use functions to override the default implementation of a
@@ -2708,19 +2708,20 @@ in the BitBake User Manual.
    to existing variables. This operator does not add any additional
    space. Also, the operator is applied after all the ``+=``, and ``=+``
    operators have been applied and after all ``=`` assignments have
-   occurred.
+   occurred. This means that if ``:append`` is used, that text can not be
+   removed.
 
    The following example shows the space being explicitly added to the
    start to ensure the appended value is not merged with the existing
    value::
 
-      SRC_URI:append = " file://fix-makefile.patch"
+      CFLAGS:append = " --enable-important-feature"
 
    You can also use
    the ``:append`` operator with overrides, which results in the actions
    only being performed for the specified target or machine::
 
-      SRC_URI:append:sh4 = " file://fix-makefile.patch"
+      CFLAGS:append:sh4 = " --enable-important-sh4-specific-feature"
 
 -  *Prepending (:prepend):* Use the ``:prepend`` operator to prepend
    values to existing variables. This operator does not add any
Signature · 1 line
2.17.1
#1 lore raw

PATCH 2/7 kernel-dev/common.rst: remove SRC_URI:append from examples

It's better to use SRC_URI += to append patches etc. If anything is added via :append, that can no longer be removed at all. If common, re-usable layers use SRC_URI:append, then users can not change those patches or SRC_URI entries without completely replacing the recipe with a copy in their own layer. Thus += is better.

---
 documentation/kernel-dev/common.rst | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
documentation/kernel-dev/common.rst +6 -6
index 16ef6453bd..fb8d7cd029 100644
@@ -360,9 +360,9 @@ home directory:
 
       FILESEXTRAPATHS:prepend := "${THISDIR}/${PN}:"
 
-      SRC_URI:append = " file://patch-file-one.patch"
-      SRC_URI:append = " file://patch-file-two.patch"
-      SRC_URI:append = " file://patch-file-three.patch"
+      SRC_URI += "file://patch-file-one.patch"
+      SRC_URI += "file://patch-file-two.patch"
+      SRC_URI += "file://patch-file-three.patch"
 
    The :term:`FILESEXTRAPATHS` and :term:`SRC_URI` statements
    enable the OpenEmbedded build system to find patch files. For more
@@ -1002,7 +1002,7 @@ Section.
    contents::
 
       FILESEXTRAPATHS:prepend := "${THISDIR}/${PN}:"
-      SRC_URI:append = " file://0001-calibrate.c-Added-some-printk-statements.patch"
+      SRC_URI += "file://0001-calibrate.c-Added-some-printk-statements.patch"
 
    The :term:`FILESEXTRAPATHS` and :term:`SRC_URI` statements
    enable the OpenEmbedded build system to find the patch file.
@@ -1875,7 +1875,7 @@ build.
 2. *Add the Feature File to SRC_URI:* Add the ``.scc`` file to the
    recipe's :term:`SRC_URI` statement::
 
-      SRC_URI:append = " file://test.scc"
+      SRC_URI += "file://test.scc"
 
    The leading space before the path is important as the path is
    appended to the existing path.
@@ -1884,7 +1884,7 @@ build.
    :term:`KERNEL_FEATURES` statement to specify the feature as a kernel
    feature::
 
-      KERNEL_FEATURES:append = " test.scc"
+      KERNEL_FEATURES += "test.scc"
 
    The OpenEmbedded build
    system processes the kernel feature when it builds the kernel.
Signature · 1 line
2.17.1
#2 lore raw

PATCH 3/7 u-boot: switch from append to += in SRC_URI

In reply to Mikko Rapeli

+= allows custom layers to change the SRC_URI e.g. when updating the whole recipe to newer u-boot version. With :append, there is no way to change the variable from a bbappend.

---
 meta/recipes-bsp/u-boot/u-boot_2022.07.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-bsp/u-boot/u-boot_2022.07.bb +1 -1
index 0d2464d74b..1ae575790c 100644
@@ -1,7 +1,7 @@
 require u-boot-common.inc
 require u-boot.inc
 
-SRC_URI:append = " file://0001-riscv32-Use-double-float-ABI-for-rv32.patch \
+SRC_URI +=       " file://0001-riscv32-Use-double-float-ABI-for-rv32.patch \
                    file://0001-riscv-fix-build-with-binutils-2.38.patch \
                  "
 
Signature · 1 line
2.17.1
#3 lore raw

PATCH 4/7 glibc-tests: use += instead of :append

In reply to Mikko Rapeli

:append can not be modified in bbappends and thus += is better in re-usable, generic layers and recipes.

---
 meta/recipes-core/glibc/glibc-tests_2.36.bb | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
meta/recipes-core/glibc/glibc-tests_2.36.bb +2 -2
index aca9675ebb..c71c0831c6 100644
@@ -4,7 +4,7 @@ require glibc-tests.inc
 inherit ptest features_check
 REQUIRED_DISTRO_FEATURES = "ptest"
 
-SRC_URI:append = " \
+SRC_URI += "\
 	file://run-ptest \
 "
 
@@ -29,7 +29,7 @@ python __anonymous() {
 RPROVIDES:${PN} = "${PN}"
 RRECOMMENDS:${PN} = ""
 RDEPENDS:${PN} = " glibc sed"
-DEPENDS:append = " sed"
+DEPENDS += "sed"
 
 export oe_srcdir="${exec_prefix}/src/debug/glibc/${PV}/"
 
Signature · 1 line
2.17.1
#4 lore raw

PATCH 5/7 go-native: switch from SRC_URI:append to SRC_URI +=

In reply to Mikko Rapeli

The :append can not be removed if needed in other layers.

---
 meta/recipes-devtools/go/go-native_1.19.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/go/go-native_1.19.bb +1 -1
index 76c0ab73a6..ddf25b2c9b 100644
@@ -5,7 +5,7 @@ require go-${PV}.inc
 
 inherit native
 
-SRC_URI:append = " https://dl.google.com/go/go1.4-bootstrap-20171003.tar.gz;name=bootstrap;subdir=go1.4"
+SRC_URI += "https://dl.google.com/go/go1.4-bootstrap-20171003.tar.gz;name=bootstrap;subdir=go1.4"
 SRC_URI[bootstrap.sha256sum] = "f4ff5b5eb3a3cae1c993723f3eab519c5bae18866b5e5f96fe1102f0cb5c3e52"
 
 export GOOS = "${BUILD_GOOS}"
Signature · 1 line
2.17.1
#5 lore raw

PATCH 6/7 python3-rfc3986-validator: switch from SRC_URI:append to SRC_URI +=

In reply to Mikko Rapeli

The :append can not be removed via bbappends if needed. Thus it's better for open source layers to use += append if possible.

---
 meta/recipes-devtools/python/python3-rfc3986-validator_0.1.1.bb | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-devtools/python/python3-rfc3986-validator_0.1.1.bb +1 -1
index 4abd181acf..e374979cb4 100644
@@ -13,7 +13,7 @@ UPSTREAM_CHECK_REGEX = "/rfc3986-validator/(?P<pver>(\d+[\.\-_]*)+)/"
 
 inherit pypi setuptools3
 
-SRC_URI:append = " \
+SRC_URI += "\
     file://0001-setup.py-move-pytest-runner-to-test_requirements.patch \
 "
 
Signature · 1 line
2.17.1
#6 lore raw

PATCH 7/7 linux-libc-headers: switch from SRC_URI:append to SRC_URI +=

In reply to Mikko Rapeli

The :append can not be removed via bbappends in custom layers so it's better to use += appends when ever possible.

---
 .../linux-libc-headers/linux-libc-headers_5.19.bb               | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/recipes-kernel/linux-libc-headers/linux-libc-headers_5.19.bb +1 -1
index 528e1d3379..3557526d14 100644
@@ -7,7 +7,7 @@ SRC_URI:append:libc-musl = "\
     file://0001-include-linux-stddef.h-in-swab.h-uapi-header.patch \
    "
 
-SRC_URI:append = "\
+SRC_URI += "\
     file://0001-scripts-Use-fixed-input-and-output-files-instead-of-.patch \
     file://0001-kbuild-install_headers.sh-Strip-_UAPI-from-if-define.patch \
 "
Signature · 1 line
2.17.1
#7 lore raw
In reply to Mikko Rapeli

Hi Mikko,

Quoted 27 lines · Mikko Rapeli

Using SRC_URI:append without recipe, machine or architecture specific limitations makes the :append'ed text unremovable and thus users and custom layers can not change the variable anymore. This makes it hard to e.g. override SRC_URI completely in a bbappend to update the full recipe to a newer version. Thus common, reusable layers which users are meant to re-use and customize should not use SRC_URI:append but SRC_URI += instead.

---
  documentation/dev-manual/common-tasks.rst | 9 +++++----
  1 file changed, 5 insertions(+), 4 deletions(-)
documentation/dev-manual/common-tasks.rst +1 -1
index b08a55331d..3112f9b893 100644
@@ -2577,7 +2577,7 @@ chapter of the BitBake User Manual.
 
        S = "${WORKDIR}/postfix-${PV}"
        CFLAGS += "-DNO_ASM"
-      SRC_URI:append = " file://fixup.patch"
+      CFLAGS:append = " --enable-important-feature"
 
  -  *Functions:* Functions provide a series of actions to be performed.
     You usually use functions to override the default implementation of a

documentation/dev-manual/common-tasks.rst in the BitBake User Manual. · line 2713

Quoted 7 lines
documentation/dev-manual/common-tasks.rst +2 -1
@@ -2708,19 +2708,20 @@ in the BitBake User Manual.
     to existing variables. This operator does not add any additional
     space. Also, the operator is applied after all the ``+=``, and ``=+``
     operators have been applied and after all ``=`` assignments have
-   occurred.
+   occurred. This means that if ``:append`` is used, that text can not be
+   removed.
 

Pedantic: it can, with :remove. One should try really hard to not use it though.

The rest is fine.

Cheers,
Quentin

#8 lore raw

Mikko, thanks for the patch series.
Quentin, thanks for the review.

Indeed, I think that the description of the commit should be improved too. See below.

Quoted 10 lines · Quentin Schulz via lists.openembedded.org

Hi Mikko,

Quoted 7 lines · Mikko Rapeli

Using SRC_URI:append without recipe, machine or architecture specific limitations makes the :append'ed text unremovable and thus users and custom layers can not change the variable anymore. This makes it hard to e.g. override SRC_URI completely in a bbappend to update the full recipe to a newer version. Thus common, reusable layers which users are meant to re-use and customize should not use SRC_URI:append but SRC_URI += instead.

What the following text instead?

Using SRC_URI:append without recipe, machine or architecture specific limitations makes the :append'ed text more difficult to override than if the "+=" operator was used. This makes it hard for example to override SRC_URI completely in a bbappend to update the full recipe to a newer version. Thus common, reusable layers which users are meant to re-use and customize should not use SRC_URI:append but SRC_URI += instead.

Quoted 6 lines
Quoted 6 lines

---
  documentation/dev-manual/common-tasks.rst | 9 +++++----

  1 file changed, 5 insertions(+), 4 deletions(-)
Quoted 9 lines
Quoted 9 lines
documentation/dev-manual/common-tasks.rst +0 -0
index b08a55331d..3112f9b893 100644
@@ -2577,7 +2577,7 @@ chapter of the BitBake User Manual.

          S = "${WORKDIR}/postfix-${PV}" CFLAGS += "-DNO_ASM"
-      SRC_URI:append = " file://fixup.patch" +      CFLAGS:append = " --enable-important-feature"

Didn't you mean
+     SRC_URI += "file://fixup.patch"

instead here?

Quoted 4 lines
Quoted 4 lines

    -  *Functions:* Functions provide a series of actions to be performed. You usually use functions to override the default implementation of a

documentation/dev-manual/common-tasks.rst +0 -0
@@ -2708,19 +2708,20 @@ in the BitBake User Manual.

     to existing variables. This operator does not add any additional

Quoted 8 lines
Quoted 6 lines

     space. Also, the operator is applied after all the ``+=``, and ``=+`` operators have been applied and after all ``=`` assignments have -   occurred.
+   occurred. This means that if ``:append`` is used, that text can not be
+   removed.

Pedantic: it can, with :remove. One should try really hard to not use it though.

What about...

"This means that if ``:append`` is used in a recipe, it cannot only be overridden by another layer using the  special ``:remove`` operator, which in turn will prevent further layers from adding it back."

I'm not very happy with this wording, because ":remove" hasn't been introduced at this stage in this section. Maybe we should add a link to https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html#conditional-syntax-overrides

Any thoughts?

Cheers
Michael.

Signature · 3 lines
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
#9 lore raw

Hi,

Quoted 20 lines · Michael Opdenacker

Mikko, thanks for the patch series.
Quentin, thanks for the review.

Indeed, I think that the description of the commit should be improved too. See below.

Quoted 10 lines · Quentin Schulz via lists.openembedded.org

Hi Mikko,

Quoted 7 lines · Mikko Rapeli

Using SRC_URI:append without recipe, machine or architecture specific limitations makes the :append'ed text unremovable and thus users and custom layers can not change the variable anymore. This makes it hard to e.g. override SRC_URI completely in a bbappend to update the full recipe to a newer version. Thus common, reusable layers which users are meant to re-use and customize should not use SRC_URI:append but SRC_URI += instead.

Quoted 8 lines

What the following text instead?

Using SRC_URI:append without recipe, machine or architecture specific limitations makes the :append'ed text more difficult to override than if the "+=" operator was used. This makes it hard for example to override SRC_URI completely in a bbappend to update the full recipe to a newer version. Thus common, reusable layers which users are meant to re-use and customize should not use SRC_URI:append but SRC_URI += instead.

Ok, will send a v2 with this.

Quoted 21 lines
Quoted 6 lines
Quoted 6 lines
---
  documentation/dev-manual/common-tasks.rst | 9 +++++----
  1 file changed, 5 insertions(+), 4 deletions(-)
Quoted 10 lines

documentation/dev-manual/common-tasks.rst chapter of the BitBake User Manual. · line 2579

Quoted 10 lines
documentation/dev-manual/common-tasks.rst +1 -1
index b08a55331d..3112f9b893 100644
@@ -2577,7 +2577,7 @@ chapter of the BitBake User Manual.
          S = "${WORKDIR}/postfix-${PV}"
        CFLAGS += "-DNO_ASM"
-      SRC_URI:append = " file://fixup.patch"
+      CFLAGS:append = " --enable-important-feature"

Didn't you mean
+ SRC_URI += "file://fixup.patch"

Quoted 1 line

instead here?

No. The sample code shows how to use variables so it needs an :append example but SRC_URI is not good use for :append's. Hence I changed it to CFLAGS:append.

Quoted 24 lines
Quoted 8 lines
Quoted 8 lines
documentation/dev-manual/common-tasks.rst +0 -0
    -  *Functions:* Functions provide a series of actions to be

performed.
You usually use functions to override the default implementation
of a

documentation/dev-manual/common-tasks.rst +0 -0
@@ -2708,19 +2708,20 @@ in the BitBake User Manual.
     to existing variables. This operator does not add any additional
     space. Also, the operator is applied after all the ``+=``, and

``=+``

Quoted 8 lines
Quoted 5 lines

operators have been applied and after all ``=`` assignments have

documentation/dev-manual/common-tasks.rst +1 -1
-   occurred.
+   occurred. This means that if ``:append`` is used, that text can

not be
+ removed.

Pedantic: it can, with :remove. One should try really hard to not use it though.

What about...

"This means that if ``:append`` is used in a recipe, it cannot only be overridden by another layer using the special ``:remove`` operator, which in turn will prevent further layers from adding it back."

Quoted 5 lines

I'm not very happy with this wording, because ":remove" hasn't been
introduced at this stage in this section. Maybe we should add a link
to https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html#conditional-syntax-overrides

Any thoughts?

I'll send v2 with this version.

I'd like to get rid of all SRC_URI:append's in documentation since that bad practice already ended in various recipes in poky master and kirkstone branches.

Cheers,
-Mikko