Skip to contentSkip to conversation

bitbake: cooker: Rename __depends in all multiconfigs

PATCH Superseded Joshua Watt 1 participant 3 messages May 21, 2019 – May 29, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 bitbake: cooker: Rename __depends in all multiconfigs

From: Joshua Watt <[email protected]>

The renaming of the __depends variable to __base_depends and file watches needs to occurs for all multiconfigs, not just the base config. Failing to do this for all multiconfigs will result in a huge increase in the size of the parsing cache (about 5x for a single mulitconfig) because all multiconfig caches will still depend on the base config files. This will also seen a similar jump in the amount of time required to load the parsing cache from memory, both because the cache is larger and because of explosion of additional existence checks that must be done for the base files.

---
 bitbake/lib/bb/cooker.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
bitbake/lib/bb/cooker.py +3 -2
index adc41014e64..78658a1683e 100644
@@ -391,8 +391,9 @@ class BBCooker:
         if CookerFeatures.BASEDATASTORE_TRACKING in self.featureset:
             self.disableDataTracking()
 
-        self.data.renameVar("__depends", "__base_depends")
-        self.add_filewatch(self.data.getVar("__base_depends", False), self.configwatcher)
+        for mc in self.databuilder.mcdata.values():
+            mc.renameVar("__depends", "__base_depends")
+            self.add_filewatch(mc.getVar("__base_depends", False), self.configwatcher)
 
         self.baseconfig_valid = True
         self.parsecache_valid = False
Signature · 1 line
2.21.0
#1 lore raw

PATCH v2 1/1 bitbake: cooker: Rename __depends in all multiconfigs

The renaming of the __depends variable to __base_depends and file watches needs to occurs for all multiconfigs, not just the base config. Failing to do this for all multiconfigs will result in a huge increase in the size of the parsing cache (about 5x for a single mulitconfig) because all multiconfig caches will still depend on the base config files. This will also seen a similar jump in the amount of time required to load the parsing cache from memory, both because the cache is larger and because of explosion of additional existence checks that must be done for the base files.

[YOCTO #13359]

---
 bitbake/lib/bb/cooker.py | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)
bitbake/lib/bb/cooker.py +3 -2
index adc41014e6..78658a1683 100644
@@ -391,8 +391,9 @@ class BBCooker:
         if CookerFeatures.BASEDATASTORE_TRACKING in self.featureset:
             self.disableDataTracking()
 
-        self.data.renameVar("__depends", "__base_depends")
-        self.add_filewatch(self.data.getVar("__base_depends", False), self.configwatcher)
+        for mc in self.databuilder.mcdata.values():
+            mc.renameVar("__depends", "__base_depends")
+            self.add_filewatch(mc.getVar("__base_depends", False), self.configwatcher)
 
         self.baseconfig_valid = True
         self.parsecache_valid = False
Signature · 1 line
2.21.0
#2 lore raw

Ping

Quoted 33 lines · Joshua Watt

The renaming of the __depends variable to __base_depends and file watches needs to occurs for all multiconfigs, not just the base config. Failing to do this for all multiconfigs will result in a huge increase in the size of the parsing cache (about 5x for a single mulitconfig) because all multiconfig caches will still depend on the base config files. This will also seen a similar jump in the amount of time required to load the parsing cache from memory, both because the cache is larger and because of explosion of additional existence checks that must be done for the base files.

[YOCTO #13359]

---
  bitbake/lib/bb/cooker.py | 5 +++--
  1 file changed, 3 insertions(+), 2 deletions(-)
bitbake/lib/bb/cooker.py +3 -2
index adc41014e6..78658a1683 100644
@@ -391,8 +391,9 @@ class BBCooker:
          if CookerFeatures.BASEDATASTORE_TRACKING in self.featureset:
              self.disableDataTracking()
 
-        self.data.renameVar("__depends", "__base_depends")
-        self.add_filewatch(self.data.getVar("__base_depends", False), self.configwatcher)
+        for mc in self.databuilder.mcdata.values():
+            mc.renameVar("__depends", "__base_depends")
+            self.add_filewatch(mc.getVar("__base_depends", False), self.configwatcher)
 
          self.baseconfig_valid = True
          self.parsecache_valid = False