Skip to contentSkip to conversation

bitbake-user-manual: Document override :append, :prepend, :remove order

PATCH Superseded Marek Vasut 3 participants 3 messages Jul 25, 2022

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 bitbake-user-manual: Document override :append, :prepend, :remove order

The application order of override syntax :append, :prepend, :remove is inobvious and undocumented. The order does not match variable parse history, i.e. output of "bitbake -e", either.

Add note documenting the exact order in which :append, :prepend, :remove are applied to variable in lib/bb/data_smart.py getVarFlag(). Add note that this order does not match the variable parse history order. This is mostly to prevent others from running into this and scratching their heads, trying to find out what is going on.

Cc Martin Jansa <[email protected]>
Cc Quentin Schulz <[email protected]>
Cc Richard Purdie <[email protected]>

---
V2: - Add RB from Quentin
- Mention un-remove implementation
- Reference :remove order in append/prepend override syntax

---
 .../bitbake-user-manual-metadata.rst          | 25 +++++++++++++++++++
 1 file changed, 25 insertions(+)
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +25 -0
index 33782161..4da18826 100644
@@ -319,6 +319,10 @@ The variable ``D`` becomes "dvaladditional data".
 
    You must control all spacing when you use the override syntax.
 
+.. note::
+
+   The overrides are applied in this order, ":append", ":prepend", ":remove".
+
 It is also possible to append and prepend to shell functions and
 BitBake-style Python functions. See the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:shell functions`" and ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:bitbake-style python functions`"
 sections for examples.
@@ -351,6 +355,27 @@ The variable ``FOO`` becomes
 Like ":append" and ":prepend", ":remove" is applied at variable
 expansion time.
 
+.. note::
+
+   The overrides are applied in this order, ":append", ":prepend", ":remove".
+   This implies it is not possible to re-append previously removed strings.
+   While it is not possible to re-append strings, it is possible to remove
+   strings from separate variable which contains the string remove list to
+   achieve the same effect using un-remove::
+
+     FOOREMOVE = "123 456 789"
+     FOO:remove = "${FOOREMOVE}"
+     FOOREMOVE:remove = "456"
+
+   This expands to ``FOO:remove = "123 789"``.
+
+.. note::
+
+   Override application order may not match variable parse history, i.e.
+   the output of ``bitbake -e`` may contain ":remove" before ":append",
+   but the result will be removed string, because ":remove" is handled
+   last.
+
 Override Style Operation Advantages
 -----------------------------------
 
Signature · 1 line
2.35.1
#1 lore raw

Hi Marek,

Quoted 40 lines · Marek Vasut

The application order of override syntax :append, :prepend, :remove is inobvious and undocumented. The order does not match variable parse history, i.e. output of "bitbake -e", either.

Add note documenting the exact order in which :append, :prepend, :remove are applied to variable in lib/bb/data_smart.py getVarFlag(). Add note that this order does not match the variable parse history order. This is mostly to prevent others from running into this and scratching their heads, trying to find out what is going on.

Cc Martin Jansa <[email protected]>
Cc Quentin Schulz <[email protected]>
Cc Richard Purdie <[email protected]>

---
V2: - Add RB from Quentin
- Mention un-remove implementation
- Reference :remove order in append/prepend override syntax

---
  .../bitbake-user-manual-metadata.rst          | 25 +++++++++++++++++++
  1 file changed, 25 insertions(+)
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +4 -0
index 33782161..4da18826 100644
@@ -319,6 +319,10 @@ The variable ``D`` becomes "dvaladditional data".
 
     You must control all spacing when you use the override syntax.
 
+.. note::
+
+   The overrides are applied in this order, ":append", ":prepend", ":remove".
+
  It is also possible to append and prepend to shell functions and
  BitBake-style Python functions. See the ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:shell functions`" and ":ref:`bitbake-user-manual/bitbake-user-manual-metadata:bitbake-style python functions`"
  sections for examples.
@@ -351,6 +355,27 @@ The variable ``FOO`` becomes
  Like ":append" and ":prepend", ":remove" is applied at variable

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst The variable ``FOO`` becomes · line 365

Quoted 10 lines
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +8 -0
  expansion time.
 
+.. note::
+
+   The overrides are applied in this order, ":append", ":prepend", ":remove".
+   This implies it is not possible to re-append previously removed strings.
+   While it is not possible to re-append strings, it is possible to remove
+   strings from separate variable which contains the string remove list to
+   achieve the same effect using un-remove::
+

I can suggest:
"""
However, one can undo a ":remove" by using an intermediate variable whose content is passed to the ":remove" so that modifying the intermediate variable equals to keeping the string in::
"""

I specifically want to avoid "un-remove". Not sure my suggestion is easier to understand as I'm usually pretty good at unwillingly writing convoluted sentences.

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst The variable ``FOO`` becomes · line 369

Quoted 5 lines
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +5 -0
+     FOOREMOVE = "123 456 789"
+     FOO:remove = "${FOOREMOVE}"
+     FOOREMOVE:remove = "456"
+ > +   This expands to ``FOO:remove = "123 789"``.
+

Did you test this? I think there's an additional white space (because only 456 gets removed but the spaces around it stay).

Cheers,
Quentin

#2 lore raw
Quoted 80 lines

-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of Quentin Schulz
Sent: den 25 juli 2022 13:48
To: Marek Vasut <[email protected]>; [email protected]
Cc: [email protected]; Quentin Schulz <[email protected]>; Martin Jansa <[email protected]>; Richard Purdie <[email protected]>
Subject: Re: [docs] [bitbake-devel] [PATCH v2] bitbake-user-manual:
Document override :append, :prepend, :remove order

Hi Marek,

Quoted 19 lines · Marek Vasut

The application order of override syntax :append, :prepend, :remove is inobvious and undocumented. The order does not match variable parse history, i.e. output of "bitbake -e", either.

Add note documenting the exact order in which :append, :prepend, :remove are applied to variable in lib/bb/data_smart.py getVarFlag(). Add note that this order does not match the variable parse history order. This is mostly to prevent others from running into this and scratching their heads, trying to find out what is going on.

Cc Martin Jansa <[email protected]>
Cc Quentin Schulz <[email protected]>
Cc Richard Purdie <[email protected]>

---
V2: - Add RB from Quentin
- Mention un-remove implementation
- Reference :remove order in append/prepend override syntax

Review

Quoted 5 lines
---
  .../bitbake-user-manual-metadata.rst          | 25 +++++++++++++++++++
  1 file changed, 25 insertions(+)
(quoted diff) +0 -0

b/doc/bitbake-user-manual/bitbake-user-manual-metadata.rst

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst The variable ``D`` becomes "dvaladditional data". · line 324

Quoted 10 lines

index 33782161..4da18826 100644

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +3 -0
@@ -319,6 +319,10 @@ The variable ``D`` becomes "dvaladditional data".
 
     You must control all spacing when you use the override syntax.
 
+.. note::
+
+   The overrides are applied in this order, ":append", ":prepend",

":remove".

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst The variable ``D`` becomes "dvaladditional data".

Quoted 3 lines
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +1 -0
+
  It is also possible to append and prepend to shell functions and
  BitBake-style Python functions. See the ":ref:`bitbake-user-

manual/bitbake-user-manual-metadata:shell functions" and ":ref:bitbake- user-manual/bitbake-user-manual-metadata:bitbake-style python functions`"

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst The variable ``FOO`` becomes · line 360

Quoted 8 lines
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +3 -0
  sections for examples.
@@ -351,6 +355,27 @@ The variable ``FOO`` becomes
  Like ":append" and ":prepend", ":remove" is applied at variable
  expansion time.
 
+.. note::
+
+   The overrides are applied in this order, ":append", ":prepend",

":remove".

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst The variable ``FOO`` becomes · line 361

Quoted 1 line
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +1 -0
+   This implies it is not possible to re-append previously removed

strings.

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst The variable ``FOO`` becomes · line 362

Quoted 1 line
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +1 -0
+   While it is not possible to re-append strings, it is possible to

remove

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst The variable ``FOO`` becomes · line 363

Quoted 1 line
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +1 -0
+   strings from separate variable which contains the string remove list

to

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst The variable ``FOO`` becomes · line 365

Quoted 2 lines
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +2 -0
+   achieve the same effect using un-remove::
+

I can suggest:
"""
However, one can undo a ":remove" by using an intermediate variable whose content is passed to the ":remove" so that modifying the intermediate variable equals to keeping the string in::
"""

Quoted 7 lines

I specifically want to avoid "un-remove". Not sure my suggestion is easier to understand as I'm usually pretty good at unwillingly writing convoluted sentences.

Quoted 3 lines
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +3 -0
+     FOOREMOVE = "123 456 789"
+     FOO:remove = "${FOOREMOVE}"
+     FOOREMOVE:remove = "456"

I recommend not using :remove here, but rather explicitly setting FOOREMOVE. While it of course works to use :remove like that, it distracts from the actual problem.

It might also be better written as:

     FOOREMOVE = "123 456 789"
     FOO:remove = "${FOOREMOVE}"

     ...

     FOOREMOVE = "123 789"

to indicate that they are typically not defined at the same place.

Quoted 8 lines

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst The variable ``FOO`` becomes · line 370

Quoted 2 lines
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +2 -0
+ > +   This expands to ``FOO:remove = "123 789"``.
+

Did you test this? I think there's an additional white space (because only 456 gets removed but the spaces around it stay).

Cheers,
Quentin

//Peter