Skip to contentSkip to conversation

data_smart: Pass flags through variable rename

PATCH In review Alex Kiernan 2 participants 6 messages Jun 9, 2022 – Jun 28, 2022 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 data_smart: Pass flags through variable rename

During variable expansion, the flags are not passed through but are discarded which makes constructs such as:

  ALTERNATIVE_TARGET:${PN}-foo[name] = "value"

fail.

This was previously attempted in 2013:

https://lists.openembedded.org/g/bitbake-devel/message/3333

It's unclear what the issue called out last time was:

Quoted 3 lines

I think this does not work for all cases but I don't know how to deal properly with it. It fixes the case of updates-alternative but breaks other code.

https://lists.openembedded.org/g/bitbake-devel/message/3334

But this code now passes both oe-selftest and bitbake-selftest.

---
 lib/bb/data_smart.py | 12 +++++++-----
 lib/bb/tests/data.py |  4 ++++
 2 files changed, 11 insertions(+), 5 deletions(-)
lib/bb/data_smart.py +7 -5
index dd20ca557ee2..308f8eab4913 100644
@@ -675,13 +675,15 @@ class DataSmart(MutableMapping):
 
         srcflags = self.getVarFlags(key, False, True) or {}
         for i in srcflags:
-            if i not in (__setvar_keyword__):
+            if i == "_content":
                 continue
             src = srcflags[i]
-
-            dest = self.getVarFlag(newkey, i, False) or []
-            dest.extend(src)
-            self.setVarFlag(newkey, i, dest, ignore=True)
+            if i in (__setvar_keyword__):
+                dest = self.getVarFlag(newkey, i, False) or []
+                dest.extend(src)
+                self.setVarFlag(newkey, i, dest, ignore=True)
+            else:
+                self.setVarFlag(newkey, i, src, ignore=True)
 
         if key in self.overridedata:
             self.overridedata[newkey] = []
lib/bb/tests/data.py +4 -0
index e667c7c7d359..5cdf1494f3c4 100644
@@ -440,6 +440,10 @@ class TestFlags(unittest.TestCase):
         self.assertEqual(self.d.getVarFlag("foo", "flag1", False), "value of flag1")
         self.assertEqual(self.d.getVarFlag("foo", "flag2", False), None)
 
+    def test_renameflag(self):
+        self.d.renameVar("foo", "bar")
+        self.assertEqual(self.d.getVarFlag("bar", "flag1", False), "value of flag1")
+
 
 class Contains(unittest.TestCase):
     def setUp(self):
Signature · 1 line
2.35.1
#1 lore raw
Quoted 4 lines · Alex Kiernan

During variable expansion, the flags are not passed through but are discarded which makes constructs such as:

  ALTERNATIVE_TARGET:${PN}-foo[name] = "value"

This is a tricky one. I don't think we've ever supported overrides and flags combined? Is this in use in metadata?

Cheers,

Richard

#2 lore raw

Not that I've found, but I'd really like to use it... I've a layer for s6-rc (https://skarnet.org/software/s6-rc/) and defining the configuration using this syntax works really well:

S6_RC_SERVICE_${BPN} ?= "longrun" S6_RC_SERVICE_${BPN}[producer-for] ?= "${BPN}-log" S6_RC_SERVICE_${BPN}-log ?= "longrun" S6_RC_SERVICE_${BPN}-log[consumer-for] ?= "${BPN}" S6_RC_SERVICE_${BPN}-log[notification-fd] ?= "3" S6_RC_SERVICE_${BPN}-log[pipeline-name] ?= "${BPN}-pipeline" S6_RC_SERVICE_${BPN}-log[run] ?= "${@ d.getVar('s6_rc_log') }"

Is the kind of thing I'd like to work.

Quoted 13 lines · Richard Purdie
Quoted 4 lines · Alex Kiernan

During variable expansion, the flags are not passed through but are discarded which makes constructs such as:

  ALTERNATIVE_TARGET:${PN}-foo[name] = "value"

This is a tricky one. I don't think we've ever supported overrides and flags combined? Is this in use in metadata?

Cheers,

Richard

Signature · 1 line
Alex Kiernan

A message that is not in the archive (sent off-list or not yet received): [email protected]

#4 lore raw

I've been thinking around this, I guess the bit I really want to work is for flags to survive through variable renames, rather than overrides, but the two use cases do tend to collide as we often have overrides with variable expansion, which is then renaming...

Just realised I c&p my example from our vendor branch which is still on _ based overrides rather than : based ones, but both seem to work for me.

Quoted 44 lines · Alex Kiernan via lists.openembedded.org

Not that I've found, but I'd really like to use it... I've a layer for s6-rc (https://skarnet.org/software/s6-rc/) and defining the configuration using this syntax works really well:

S6_RC_SERVICE_${BPN} ?= "longrun" S6_RC_SERVICE_${BPN}[producer-for] ?= "${BPN}-log" S6_RC_SERVICE_${BPN}-log ?= "longrun" S6_RC_SERVICE_${BPN}-log[consumer-for] ?= "${BPN}" S6_RC_SERVICE_${BPN}-log[notification-fd] ?= "3" S6_RC_SERVICE_${BPN}-log[pipeline-name] ?= "${BPN}-pipeline" S6_RC_SERVICE_${BPN}-log[run] ?= "${@ d.getVar('s6_rc_log') }"

Is the kind of thing I'd like to work.

Quoted 5 lines · Richard Purdie
Quoted 3 lines · Alex Kiernan

During variable expansion, the flags are not passed through but are discarded which makes constructs such as:

Quoted 8 lines
Quoted 1 line
  ALTERNATIVE_TARGET:${PN}-foo[name] = "value"

This is a tricky one. I don't think we've ever supported overrides and flags combined? Is this in use in metadata?

Cheers,

Richard

--
Alex Kiernan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#13747): https://lists.openembedded.org/g/bitbake-devel/message/13747
Mute This Topic: https://lists.openembedded.org/mt/91640731/3618097
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-

Signature · 1 line
Alex Kiernan
#5 lore raw
Quoted 8 lines · Alex Kiernan

I've been thinking around this, I guess the bit I really want to work is for flags to survive through variable renames, rather than overrides, but the two use cases do tend to collide as we often have overrides with variable expansion, which is then renaming...

Just realised I c&p my example from our vendor branch which is still on _ based overrides rather than : based ones, but both seem to work for me.

I've been pondering this a bit more but I'm not sure we want to wade into the complexities of how flags+overrides+key expansion can all collide together.

I did notice this in the metadata though:

RRECOMMENDS:coreutils-dev[nodeprrecs] = "1"

which happens to work as it doesn't use key expansion and doesn't rely on override expansion for access (the variable named "RRECOMMENDS:coreutils-dev" exists).

It does mean we have some subset of this problem live in metadata though :/.

Changing it to RRECOMMENDS:${PN}-dev breaks things.

Cheers,

Richard

#6 lore raw
Quoted 7 lines · Richard Purdie
Quoted 5 lines · Alex Kiernan

I've been thinking around this, I guess the bit I really want to work is for flags to survive through variable renames, rather than overrides, but the two use cases do tend to collide as we often have overrides with variable expansion, which is then renaming...

Quoted 8 lines
Quoted 3 lines

Just realised I c&p my example from our vendor branch which is still on _ based overrides rather than : based ones, but both seem to work for me.

I've been pondering this a bit more but I'm not sure we want to wade into the complexities of how flags+overrides+key expansion can all collide together.

I guess I understand this, even if I'd like it to work :)

Quoted 4 lines

I did notice this in the metadata though:

RRECOMMENDS:coreutils-dev[nodeprrecs] = "1"

Quoted 9 lines

which happens to work as it doesn't use key expansion and doesn't rely on override expansion for access (the variable named "RRECOMMENDS:coreutils-dev" exists).

It does mean we have some subset of this problem live in metadata though :/.

Changing it to RRECOMMENDS:${PN}-dev breaks things.

Feels like it merits at least a warning? Though I guess the challenge is chasing an ever moving set of internal flags which can be attached to a variable.

Signature · 1 line
Alex Kiernan