Skip to contentSkip to conversation

Store a list of parsed configuration files

PATCH In review Joshua Lock 2 participants 6 messages Aug 2, 2011 – Aug 8, 2011

lore
Share & tools
lore

Joshua Lock

author
#0 lore raw

cover Store a list of parsed configuration files

This series addresses Yocto #1246 "Configuration changes could be saved to a file the cooker isn't using" by having the parser store a list of configuration files it has parsed in the data object and then switching the cooker to verify against this list that the found file was parsed before emitting the ConfigFilePathFound event.

The following changes since commit 1009ca570a750a00b0e60afcc30ead070c7b310a:

hob: remove temporary directory on program shutdown (2011-07-30 12:21:18 -0700)

are available in the git repository at:
git://github.com/incandescant/bitbake configfiles
https://github.com/incandescant/bitbake/tree/configfiles

Joshua Lock (2):
bb/parse: track parsed configuration files in __BB_PARSED_CONF bb/cooker: only emit ConfigFilePathFound for files which were parsed

 lib/bb/cooker.py                     |   18 +++++++++++++++++-
 lib/bb/parse/__init__.py             |   12 +++++++++++-
 lib/bb/parse/parse_py/ConfHandler.py |    2 ++
 3 files changed, 30 insertions(+), 2 deletions(-)
Signature · 1 line
1.7.6
author
#1 lore raw

PATCH 1/2 bb/parse: track parsed configuration files in __BB_PARSED_CONF

Keep a list of each configuration (.conf) file which is handled by the ConfHandler in the internal __BB_PARSED_CONF variable.

Signed-off-by Joshua Lock <[email protected]>
---
 lib/bb/parse/__init__.py             |   12 +++++++++++-
 lib/bb/parse/parse_py/ConfHandler.py |    2 ++
 2 files changed, 13 insertions(+), 1 deletions(-)
lib/bb/parse/__init__.py +11 -1
index eee8d9c..afe7681 100644
@@ -59,13 +59,23 @@ def update_mtime(f):
     __mtime_cache[f] = os.stat(f)[stat.ST_MTIME]
     return __mtime_cache[f]
 
-def mark_dependency(d, f):
+def full_path(f):
     if f.startswith('./'):
         f = "%s/%s" % (os.getcwd(), f[2:])
+    return f
+
+def mark_dependency(d, f):
+    full_path(f)
     deps = bb.data.getVar('__depends', d) or set()
     deps.update([(f, cached_mtime(f))])
     bb.data.setVar('__depends', deps, d)
 
+def track_conf(d, f):
+    full_path(f)
+    conf = bb.data.getVar('__BB_PARSED_CONF', d) or []
+    conf.append(f)
+    bb.data.setVar('__BB_PARSED_CONF', conf, d)
+
 def supports(fn, data):
     """Returns true if we have a handler for this file, false otherwise"""
     for h in handlers:
lib/bb/parse/parse_py/ConfHandler.py +2 -0
index 102c0e9..953ef87 100644
@@ -85,6 +85,8 @@ def handle(fn, data, include):
     if include:
         bb.parse.mark_dependency(data, abs_fn)
 
+    bb.parse.track_conf(data, abs_fn)
+
     statements = ast.StatementGroup()
     lineno = 0
     while True:
Signature · 1 line
1.7.6
author
#2 lore raw

PATCH 2/2 bb/cooker: only emit ConfigFilePathFound for files which were parsed

In reply to Joshua Lock

When the requested configuration file is found on disk check the configuration object to ensure the file was parsed before emitting the ConfigFilePathFound event. If it wasn't just return (and don't emit).

Fixes [YOCTO #1246]

Signed-off-by Joshua Lock <[email protected]>
---
 lib/bb/cooker.py |   18 +++++++++++++++++-
 1 files changed, 17 insertions(+), 1 deletions(-)
lib/bb/cooker.py +17 -1
index 025dfe4..a3456c3 100644
@@ -609,9 +609,25 @@ class BBCooker:
                 collectlog.warn("No bb files matched BBFILE_PATTERN_%s '%s'" % (collection, pattern))
 
     def findConfigFilePath(self, configfile):
+        """
+        Find the location on disk of configfile and if it exists and was parsed by BitBake
+        emit the ConfigFilePathFound event with the path to the file.
+        """
         path = self._findConfigFile(configfile)
-        if path:
+        if not path:
+            return
+
+        parsed_confs = bb.data.getVar('__BB_PARSED_CONF', self.configuration.data) or []
+        if path and path in parsed_confs:
             bb.event.fire(bb.event.ConfigFilePathFound(path), self.configuration.data)
+        elif path:
+            _, conf, conffile = path.rpartition("conf/")
+            match = os.path.join(conf, conffile)
+            # Try and find matches for conf/conffilename.conf
+            for cfg in parsed_confs:
+                if cfg.endswith(match):
+                    bb.event.fire(bb.event.ConfigFilePathFound(path), self.configuration.data)
+                    break
 
     def findFilesMatchingInDir(self, filepattern, directory):
         """
Signature · 1 line
1.7.6
#3 lore raw
In reply to Joshua Lock
Quoted 10 lines · Joshua Lock

This series addresses Yocto #1246 "Configuration changes could be saved to a file the cooker isn't using" by having the parser store a list of configuration files it has parsed in the data object and then switching the cooker to verify against this list that the found file was parsed before emitting the ConfigFilePathFound event.

The following changes since commit 1009ca570a750a00b0e60afcc30ead070c7b310a:

hob: remove temporary directory on program shutdown (2011-07-30 12:21:18 -0700)

Quoted 7 lines

are available in the git repository at:
git://github.com/incandescant/bitbake configfiles
https://github.com/incandescant/bitbake/tree/configfiles

Joshua Lock (2):
bb/parse: track parsed configuration files in __BB_PARSED_CONF bb/cooker: only emit ConfigFilePathFound for files which were parsed

Am I right in thinking this series is now superseded by the tweak to include the information in __base_depends?

Cheers,

Richard

author
#4 lore raw
Quoted 15 lines · Richard Purdie
Quoted 14 lines · Joshua Lock

This series addresses Yocto #1246 "Configuration changes could be saved to a file the cooker isn't using" by having the parser store a list of configuration files it has parsed in the data object and then switching the cooker to verify against this list that the found file was parsed before emitting the ConfigFilePathFound event.

The following changes since commit 1009ca570a750a00b0e60afcc30ead070c7b310a:

hob: remove temporary directory on program shutdown (2011-07-30 12:21:18 -0700)

are available in the git repository at:
git://github.com/incandescant/bitbake configfiles
https://github.com/incandescant/bitbake/tree/configfiles

Quoted 6 lines
Quoted 3 lines

Joshua Lock (2):
bb/parse: track parsed configuration files in __BB_PARSED_CONF bb/cooker: only emit ConfigFilePathFound for files which were parsed

Am I right in thinking this series is now superseded by the tweak to include the information in __base_depends?

That's right. The only reason I haven't sent that patch out yet is because I'm in two minds about which is the cleanest way.

Because __base_depends a) contains more than just .conf files and b) gets renamed I'm thinking the cleanest thing to do would be to iterate __depends before the call to renameVar, pull out all conf files and store them in self.configuration.config_files (or similar).

I wasn't sure adding an extra list to the configuration object would be appreciated though...

Cheers,
Joshua

Signature · 3 lines
Joshua Lock
        Yocto Project "Johannes factotum"
        Intel Open Source Technology Centre
#5 lore raw
Quoted 15 lines · Joshua Lock
Quoted 10 lines · Richard Purdie
Quoted 6 lines · Joshua Lock

git://github.com/incandescant/bitbake configfiles
https://github.com/incandescant/bitbake/tree/configfiles

Joshua Lock (2):
bb/parse: track parsed configuration files in __BB_PARSED_CONF bb/cooker: only emit ConfigFilePathFound for files which were parsed

Am I right in thinking this series is now superseded by the tweak to include the information in __base_depends?

That's right. The only reason I haven't sent that patch out yet is because I'm in two minds about which is the cleanest way.

Quoted 7 lines

Because __base_depends a) contains more than just .conf files and b) gets renamed I'm thinking the cleanest thing to do would be to iterate __depends before the call to renameVar, pull out all conf files and store them in self.configuration.config_files (or similar).

I wasn't sure adding an extra list to the configuration object would be appreciated though...

I'm not sure we need a variable just for .conf files. Anyone wanting conf files can generate that information easily enough...

Cheers,

Richard