Skip to contentSkip to conversation

bitbake-user-manual: Document override :append, :prepend, :remove order

PATCH Superseded Marek Vasut 3 participants 5 messages Jul 25, 2022 – Aug 19, 2022

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH v3 1/1 bitbake-user-manual: Document override :append, :prepend, :remove order

The application order of override syntax :append, :prepend, :remove is inobvious and undocumented. The order does not match variable parse history, i.e. output of "bitbake -e", either.

Add note documenting the exact order in which :append, :prepend, :remove are applied to variable in lib/bb/data_smart.py getVarFlag(). Add note that this order does not match the variable parse history order. This is mostly to prevent others from running into this and scratching their heads, trying to find out what is going on.

Cc Martin Jansa <[email protected]>
Cc Peter Kjellerstedt <[email protected]>
Cc Quentin Schulz <[email protected]>
Cc Richard Purdie <[email protected]>

---
V2: - Add RB from Quentin
- Mention un-remove implementation
- Reference :remove order in append/prepend override syntax
V3: - Reword the note per suggestion from Quentin
- Use explicit FOOREMOVE per suggestion from Peter

---
 .../bitbake-user-manual-metadata.rst          | 26 +++++++++++++++++++
 1 file changed, 26 insertions(+)
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +26 -0
index 33782161..587c9120 100644
@@ -319,6 +319,10 @@ The variable ``D`` becomes "dvaladditional data".
 
    You must control all spacing when you use the override syntax.
 
+.. note::
+
+   The overrides are applied in this order, ":append", ":prepend", ":remove".
+
 It is also possible to append and prepend to shell functions and
 BitBake-style Python functions. See the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:shell functions`" and ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:bitbake-style python functions`"
 sections for examples.
@@ -351,6 +355,28 @@ The variable ``FOO`` becomes
 Like ":append" and ":prepend", ":remove" is applied at variable
 expansion time.
 
+.. note::
+
+   The overrides are applied in this order, ":append", ":prepend", ":remove".
+   This implies it is not possible to re-append previously removed strings.
+   However, one can undo a ":remove" by using an intermediate variable whose
+   content is passed to the ":remove" so that modifying the intermediate
+   variable equals to keeping the string in::
+
+     FOOREMOVE = "123 456 789"
+     FOO:remove = "${FOOREMOVE}"
+     ...
+     FOOREMOVE = "123 789"
+
+   This expands to ``FOO:remove = "123 789"``.
+
+.. note::
+
+   Override application order may not match variable parse history, i.e.
+   the output of ``bitbake -e`` may contain ":remove" before ":append",
+   but the result will be removed string, because ":remove" is handled
+   last.
+
 Override Style Operation Advantages
 -----------------------------------
 
Signature · 1 line
2.35.1
#1 lore raw

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst The variable ``D`` becomes "dvaladditional data".

Quoted 6 lines · Marek Vasut

+.. note::

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +3 -0
+
+   The overrides are applied in this order, ":append", ":prepend", ":remove".
+   This implies it is not possible to re-append previously removed
    strings.

fwiw, this is possible by anonymous python functions which are evaluated after ":remove"

| FOO = "123 abc XYZ"
| 
| python () {
|     v = d.getVar('FOO')
|     d.setVar('FOO', v + ' abc 789')
| }
| 
| FOO:remove = "abc XYZ"

results in

| FOO="123 abc 789"

Enrico

#2 lore raw
Quoted 21 lines · Enrico Scholz

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst The variable ``D`` becomes "dvaladditional data".

Quoted 6 lines · Marek Vasut

+.. note::

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +3 -0
+
+   The overrides are applied in this order, ":append", ":prepend", ":remove".
+   This implies it is not possible to re-append previously removed
     strings.

fwiw, this is possible by anonymous python functions which are evaluated after ":remove"

| FOO = "123 abc XYZ"
|
| python () {
|     v = d.getVar('FOO')
|     d.setVar('FOO', v + ' abc 789')
| }
|
| FOO:remove = "abc XYZ"
Quoted 4 lines

results in

| FOO="123 abc 789"

IIRC anonymous python functions are frowned upon, aren't they ?

#3 lore raw
In reply to Enrico Scholz
Quoted 14 lines · Enrico Scholz via lists.yoctoproject.org

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst The variable ``D`` becomes "dvaladditional data".

Quoted 6 lines · Marek Vasut

+.. note::

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +3 -0
+
+   The overrides are applied in this order, ":append", ":prepend", ":remove".
+   This implies it is not possible to re-append previously removed
    strings.

fwiw, this is possible by anonymous python functions which are evaluated after ":remove"

Quoted 2 lines

FOO = "123 abc XYZ"

Quoted 10 lines
Quoted 6 lines
python () {
    v = d.getVar('FOO')
    d.setVar('FOO', v + ' abc 789')
}

FOO:remove = "abc XYZ"

results in

Quoted 1 line

FOO="123 abc 789"

The reason for this isn't perhaps obvious and worth documenting - use of setVar clears all append/prepend/remove operations against a variable.

FOO = "A"
FOO:append = " B"

python () {
    d.setVar('FOO', 'C')
}

results in C.

Cheers,

Richard

#4 lore raw
In reply to Marek Vasut
Quoted 9 lines · Marek Vasut

The application order of override syntax :append, :prepend, :remove is inobvious and undocumented. The order does not match variable parse history, i.e. output of "bitbake -e", either.

Add note documenting the exact order in which :append, :prepend, :remove are applied to variable in lib/bb/data_smart.py getVarFlag(). Add note that this order does not match the variable parse history order. This is mostly to prevent others from running into this and scratching their heads, trying to find out what is going on.

Can this be applied now or is there something more to change ?