Skip to contentSkip to conversation

bitbake-user-manual: Document override :append, :prepend, :remove order

PATCH Superseded Marek Vasut 3 participants 4 messages Jul 23, 2022 – Jul 25, 2022

v1 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 bitbake-user-manual: Document override :append, :prepend, :remove order

The application order of override syntax :append, :prepend, :remove is inobvious and undocumented. The order does not match variable parse history, i.e. output of "bitbake -e", either.

Add note documenting the exact order in which :append, :prepend, :remove are applied to variable in lib/bb/data_smart.py getVarFlag(). Add note that this order does not match the variable parse history order. This is mostly to prevent others from running into this and scratching their heads, trying to find out what is going on.

Cc Richard Purdie <[email protected]>
---
 .../bitbake-user-manual-metadata.rst                 | 12 ++++++++++++
 1 file changed, 12 insertions(+)
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +12 -0
index 33782161..651116ce 100644
@@ -351,6 +351,18 @@ The variable ``FOO`` becomes
 Like ":append" and ":prepend", ":remove" is applied at variable
 expansion time.
 
+.. note::
+
+   The overrides are applied in this order, ":append", ":prepend", ":remove".
+   This implies it is not possible to re-append previously removed strings.
+
+.. note::
+
+   Override application order may not match variable parse history, i.e.
+   the output of ``bitbake -e`` may contain ":remove" before ":append",
+   but the result will be removed string, because ":remove" is handled
+   last.
+
 Override Style Operation Advantages
 -----------------------------------
 
Signature · 1 line
2.35.1
Reviewed-by on v1 1/1
#1 lore raw

Hi Marek,

Quoted 26 lines · Marek Vasut

The application order of override syntax :append, :prepend, :remove is inobvious and undocumented. The order does not match variable parse history, i.e. output of "bitbake -e", either.

Add note documenting the exact order in which :append, :prepend, :remove are applied to variable in lib/bb/data_smart.py getVarFlag(). Add note that this order does not match the variable parse history order. This is mostly to prevent others from running into this and scratching their heads, trying to find out what is going on.

Cc Richard Purdie <[email protected]>
---
  .../bitbake-user-manual-metadata.rst                 | 12 ++++++++++++
  1 file changed, 12 insertions(+)
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +2 -0
index 33782161..651116ce 100644
@@ -351,6 +351,18 @@ The variable ``FOO`` becomes
  Like ":append" and ":prepend", ":remove" is applied at variable
  expansion time.
 
+.. note::
+

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst The variable ``FOO`` becomes · line 365

Quoted 10 lines
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +10 -0
+   The overrides are applied in this order, ":append", ":prepend", ":remove".
+   This implies it is not possible to re-append previously removed strings.
+
+.. note::
+
+   Override application order may not match variable parse history, i.e.
+   the output of ``bitbake -e`` may contain ":remove" before ":append",
+   but the result will be removed string, because ":remove" is handled
+   last.
+

I can suggest a more concise note, since I don't think we care about the actual order of append and prepend:

```
``:remove`` is final. Any string appearing in ``:remove`` cannot be added back regardless of when ``:remove`` was parsed.
```

Up to you :)

Otherwise, welcome addition thanks :)

I'm surprised this was not mentioned in https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html#removal-override-style-syntax and https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html#appending-and-prepending-override-style-syntax, so adding a few words there would be great too :)

Cheers,
Quentin

Quoted 14 lines
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +0 -1
  Override Style Operation Advantages
  -----------------------------------
 
 
 
 
-=-=-=-=-=-=-=-=-=-=-=-

Links: You receive all messages sent to this group.
View/Reply Online (#13837): https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_g_bitbake-2Ddevel_message_13837&d=DwIFaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=rAdWPF-nAT2ooJM9hyieW5jHTZRB2rlSJfhuxJeBaS29jNG06J_Qzv4xTP81zwUH&s=SzPrWDNzoIr-MjT-x6vXbW8rxAt0l0odSFygeF4NjTE&e=
Mute This Topic: https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_mt_92560243_6293953&d=DwIFaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=rAdWPF-nAT2ooJM9hyieW5jHTZRB2rlSJfhuxJeBaS29jNG06J_Qzv4xTP81zwUH&s=UaYe1Ue7jye5Pir7dZS1iHXBrTN_mWaZkOBIxibHJHA&e=
Group Owner: [email protected]
Unsubscribe: https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.openembedded.org_g_bitbake-2Ddevel_unsub&d=DwIFaQ&c=_sEr5x9kUWhuk4_nFwjJtA&r=LYjLexDn7rXIzVmkNPvw5ymA1XTSqHGq8yBP6m6qZZ4njZguQhZhkI_-172IIy1t&m=rAdWPF-nAT2ooJM9hyieW5jHTZRB2rlSJfhuxJeBaS29jNG06J_Qzv4xTP81zwUH&s=qYGfFmF6YZwURBbgyGUOFrulcscOXAZUk22WrexXNA8&e= [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

#2 lore raw
Quoted 1 line · Quentin Schulz

Hi Marek,

Hi,

Quoted 40 lines · Marek Vasut
Quoted 29 lines

The application order of override syntax :append, :prepend, :remove is inobvious and undocumented. The order does not match variable parse history, i.e. output of "bitbake -e", either.

Add note documenting the exact order in which :append, :prepend, :remove are applied to variable in lib/bb/data_smart.py getVarFlag(). Add note that this order does not match the variable parse history order. This is mostly to prevent others from running into this and scratching their heads, trying to find out what is going on.

Cc Richard Purdie <[email protected]>

---
  .../bitbake-user-manual-metadata.rst                 | 12 ++++++++++++

  1 file changed, 12 insertions(+)
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +0 -0
index 33782161..651116ce 100644
@@ -351,6 +351,18 @@ The variable ``FOO`` becomes

  Like ":append" and ":prepend", ":remove" is applied at variable expansion time.
+.. note::
+
+   The overrides are applied in this order, ":append", ":prepend", ":remove".
+   This implies it is not possible to re-append previously removed strings.

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst The variable ``FOO`` becomes · line 365

Quoted 8 lines
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +8 -0
+
+.. note::
+
+   Override application order may not match variable parse history, i.e.
+   the output of ``bitbake -e`` may contain ":remove" before ":append",
+   but the result will be removed string, because ":remove" is handled
+   last.
+

I can suggest a more concise note, since I don't think we care about the actual order of append and prepend:

Quoted 4 lines
```
``:remove`` is final. Any string appearing in ``:remove`` cannot be added back regardless of when ``:remove`` was parsed.
```

Up to you :)

I'd argue it's better to keep the information full and complete, esp. since its not easy to find within bitbake sources.

Quoted 4 lines

Otherwise, welcome addition thanks :)

I'm surprised this was not mentioned in https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html#removal-override-style-syntax and https://docs.yoctoproject.org/bitbake/bitbake-user-manual/bitbake-user-manual-metadata.html#appending-and-prepending-override-style-syntax, so adding a few words there would be great too :)

For appending and prepending, this makes little difference. It's the remove which has this nasty catch.

#3 lore raw
Quoted 54 lines · Marek Vasut
Quoted 1 line · Quentin Schulz

Hi Marek,

Hi,

Quoted 27 lines · Marek Vasut
Quoted 26 lines

The application order of override syntax :append, :prepend, :remove is inobvious and undocumented. The order does not match variable parse history, i.e. output of "bitbake -e", either.

Add note documenting the exact order in which :append, :prepend, :remove are applied to variable in lib/bb/data_smart.py getVarFlag(). Add note that this order does not match the variable parse history order. This is mostly to prevent others from running into this and scratching their heads, trying to find out what is going on.

Cc Richard Purdie <[email protected]>
---
  .../bitbake-user-manual-metadata.rst                 | 12 ++++++++++++
  1 file changed, 12 insertions(+)
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +2 -0
index 33782161..651116ce 100644
@@ -351,6 +351,18 @@ The variable ``FOO`` becomes
  Like ":append" and ":prepend", ":remove" is applied at variable
  expansion time.
+.. note::
+
Quoted 8 lines

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst The variable ``FOO`` becomes

Quoted 8 lines
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +1 -0
+   The overrides are applied in this order, ":append", ":prepend",

":remove".
+ This implies it is not possible to re-append previously removed
strings.

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +4 -0
+
+.. note::
+
+   Override application order may not match variable parse history,

i.e.

Quoted 5 lines

doc/bitbake-user-manual/bitbake-user-manual-metadata.rst The variable ``FOO`` becomes · line 365

Quoted 4 lines
doc/bitbake-user-manual/bitbake-user-manual-metadata.rst +4 -0
+   the output of ``bitbake -e`` may contain ":remove" before ":append",
+   but the result will be removed string, because ":remove" is handled
+   last.
+
Quoted 7 lines

I can suggest a more concise note, since I don't think we care about the actual order of append and prepend:

```
``:remove`` is final. Any string appearing in ``:remove`` cannot be
added back regardless of when ``:remove`` was parsed.
```

Up to you :)

Quoted 3 lines

I'd argue it's better to keep the information full and complete, esp. since its not easy to find within bitbake sources.

Then I would suggest also to mention a way to undo :remove with separate variable as discussed on IRC.

Cheers,