cooker: fixes and parsing performance improve
cover cooker: fixes and parsing performance improve
Hi RP,
These patches contain fixes for memory resident bitbake, and parsing/loading improvement, please see the patches for details.
BTW., do we have a plan to switch to memory resident bitbake by default? I've tested it a lot with these patches, it works well.
// Robert
The following changes since commit 385944254d0ef88bd6450a221a54cbcb40020b42:
tcmode-default.inc: drop preferred version of gzip-native (2018-01-31 17:01:20 +0000)
are available in the git repository at:
git://git.pokylinux.org/poky-contrib rbt/inotify
http://git.pokylinux.org/cgit.cgi//log/?h=rbt/inotify
Robert Yang (8):
bitbake: cooker: skip when BBFILE_PATTERN is empty
bitbake: cooker: fix for BBFILE_PATTERN matches bbappend
bitbake: cooker: don't stop file notifier when cooker is shutdown
bitbake: cooker: don't preserve BB_CONSOLELOG
bitbake: cache: improve debug message
bitbake: parse: fixes for resolve_file()
bitbake: cooker: fix event notifications
bitbake: cooker: improve inotify handling
bitbake/lib/bb/cache.py | 14 +++-- bitbake/lib/bb/cooker.py | 79 +++++++++++++++------------- bitbake/lib/bb/parse/__init__.py | 3 +- bitbake/lib/bb/parse/parse_py/BBHandler.py | 3 -- bitbake/lib/bb/parse/parse_py/ConfHandler.py | 3 -- bitbake/lib/bb/server/process.py | 2 + 6 files changed, 55 insertions(+), 49 deletions(-)
Signature · 1 line
2.7.4
PATCH 1/8 bitbake: cooker: skip when BBFILE_PATTERN is empty
There is nothing to do when BBFILE_PATTERN is empty.
--- bitbake/lib/bb/cooker.py | 1 + 1 file changed, 1 insertion(+)
bitbake/lib/bb/cooker.py +1 -0
@@ -1170,6 +1170,7 @@ class BBCooker: elif regex == "": parselog.debug(1, "BBFILE_PATTERN_%s is empty" % c) errors = False + continue else: try: cre = re.compile(regex)
Signature · 1 line
2.7.4
PATCH 2/8 bitbake: cooker: fix for BBFILE_PATTERN matches bbappend
In reply to Robert YangThe old code couldn't handle nestled layers correctly, e.g.:
parent_layer/sub_layer/foo.bb
Note there are two layers, parent_layer and sub_layer. And in parent_layer/conf/layer.conf:BBFILE_PATTERN_parent_layer = ""^${LAYERDIR}/"
This setting is incorrect since it also matches parent_layer/sub_layer/foo.bb, so it warns that no files matched sub_layer, this is the expected behavior, but it doesn't warn when there is a parent_layer/sub_layer/bar.bbappend, this was incorrect since the bbappend is also matched by BBFILE_PATTERN_parent_layer, it should warn and let the user fix the problem. Check the bbappend in already "matched set" before return it as matched by "unmatched set" can fix the problem.
--- bitbake/lib/bb/cooker.py | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
bitbake/lib/bb/cooker.py +7 -3
@@ -1808,21 +1808,25 @@ class CookerCollectFiles(object): realfn, cls, mc = bb.cache.virtualfn2realfn(p) priorities[p] = self.calc_bbfile_priority(realfn, matched) - # Don't show the warning if the BBFILE_PATTERN did match .bbappend files unmatched = set() for _, _, regex, pri in self.bbfile_config_priorities: if not regex in matched: unmatched.add(regex) - def findmatch(regex): + # Don't show the warning if the BBFILE_PATTERN did match .bbappend files + def find_bbappend_match(regex): for b in self.bbappends: (bbfile, append) = b if regex.match(append): + # If the bbappend is matched by already "matched set", return False + for matched_regex in matched: + if matched_regex.match(append): + return False return True return False for unmatch in unmatched.copy(): - if findmatch(unmatch): + if find_bbappend_match(unmatch): unmatched.remove(unmatch) for collection, pattern, regex, _ in self.bbfile_config_priorities:
Signature · 1 line
2.7.4
PATCH 3/8 bitbake: cooker: don't stop file notifier when cooker is shutdown
In reply to Robert YangIt should be live/exited with server rather than cooker, fixed:
$ bitbake --server-only -T -1 Set MACHINE = "qemux86" in conf/local.conf $ bitbake quilt Set MACHINE = "qemuppc" in conf/local.conf $ bitbake quilt [snip] ERROR: When reparsing /workspace1/lyang1/poky/meta/recipes-connectivity/openssl/openssl_1.0.2m.bb.do_package, the basehash value changed from c216f7f4fdd3cf4a0b10b975a636426c to d5a8e9431ab261381752d7a64c7b2fa9. The metadata is not deterministic and this needs to be fixed. [snip]
This is because the server doesn't know local.conf is changed since the notifiers are stopped, so it doesn't reparse, and then we would get the errors, let the notifiers live/exited with server can fix the problem.
--- bitbake/lib/bb/cooker.py | 2 -- bitbake/lib/bb/server/process.py | 2 ++ 2 files changed, 2 insertions(+), 2 deletions(-)
bitbake/lib/bb/cooker.py +0 -2
@@ -1604,8 +1604,6 @@ class BBCooker: if self.parser: self.parser.shutdown(clean=not force, force=force) - self.notifier.stop() - self.confignotifier.stop() def finishcommand(self): self.state = state.initial
bitbake/lib/bb/server/process.py +2 -0
@@ -223,6 +223,8 @@ class ProcessServer(multiprocessing.Process): try: self.cooker.shutdown(True) + self.cooker.notifier.stop() + self.cooker.confignotifier.stop() except: pass
Signature · 1 line
2.7.4
PATCH 4/8 bitbake: cooker: don't preserve BB_CONSOLELOG
In reply to Robert YangFixed:
$ bitbake --server-only -T -1 Set MACHINE = "qemux86" in conf/local.conf $ bitbake quilt Set MACHINE = "qemuppc" in conf/local.conf $ bitbake quilt
The log still goes into tmp/log/cooker/qemux86 in the second run, this is incorrect (should be tmp/log/cooker/qemuppc). I checked the code, the ui initializes it every time when it starts, so let it use the up-to-date one rather than old one.
--- bitbake/lib/bb/cooker.py | 8 -------- 1 file changed, 8 deletions(-)
bitbake/lib/bb/cooker.py +0 -8
@@ -326,11 +326,6 @@ class BBCooker: self.state = state.initial self.caches_array = [] - # Need to preserve BB_CONSOLELOG over resets - consolelog = None - if hasattr(self, "data"): - consolelog = self.data.getVar("BB_CONSOLELOG") - if CookerFeatures.BASEDATASTORE_TRACKING in self.featureset: self.enableDataTracking()
@@ -358,9 +353,6 @@ class BBCooker: self.data_hash = self.databuilder.data_hash self.extraconfigdata = {} - if consolelog: - self.data.setVar("BB_CONSOLELOG", consolelog) - self.data.setVar('BB_CMDLINE', self.ui_cmdline) #
Signature · 1 line
2.7.4
PATCH 5/8 bitbake: cache: improve debug message
In reply to Robert Yang* Print message when cachefile is found/not can help debug.
* Update "Using cache in" to "Cache dir:" since it was the same as the debug message of "codeparser & file checksum caches", which caused confusion. And whether the cache file will be used or not is still unknown at that time, so just print the cache dir.
--- bitbake/lib/bb/cache.py | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
bitbake/lib/bb/cache.py +4 -1
@@ -395,7 +395,7 @@ class Cache(NoCache): self.has_cache = True self.cachefile = getCacheFile(self.cachedir, "bb_cache.dat", self.data_hash) - logger.debug(1, "Using cache in '%s'", self.cachedir) + logger.debug(1, "Cache dir: %s", self.cachedir) bb.utils.mkdirhier(self.cachedir) cache_ok = True
@@ -408,6 +408,8 @@ class Cache(NoCache): self.load_cachefile() elif os.path.isfile(self.cachefile): logger.info("Out of date cache found, rebuilding...") + else: + logger.debug(1, "Cache file %s not found, building..." % self.cachefile) def load_cachefile(self): cachesize = 0
@@ -424,6 +426,7 @@ class Cache(NoCache): for cache_class in self.caches_array: cachefile = getCacheFile(self.cachedir, cache_class.cachefile, self.data_hash) + logger.debug(1, 'Loading cache file: %s' % cachefile) with open(cachefile, "rb") as cachefile: pickled = pickle.Unpickler(cachefile) # Check cache version information
Signature · 1 line
2.7.4
PATCH 6/8 bitbake: parse: fixes for resolve_file()
In reply to Robert YangThe resolve_file() calls mark_dependency(), so the one which calls resolve_file() doesn't need call mark_dependency() again.
--- bitbake/lib/bb/parse/__init__.py | 3 ++- bitbake/lib/bb/parse/parse_py/BBHandler.py | 3 --- bitbake/lib/bb/parse/parse_py/ConfHandler.py | 3 --- 3 files changed, 2 insertions(+), 7 deletions(-)
bitbake/lib/bb/parse/__init__.py +2 -1
@@ -134,8 +134,9 @@ def resolve_file(fn, d): if not newfn: raise IOError(errno.ENOENT, "file %s not found in %s" % (fn, bbpath)) fn = newfn + else: + mark_dependency(d, fn) - mark_dependency(d, fn) if not os.path.isfile(fn): raise IOError(errno.ENOENT, "file %s not found" % fn)
bitbake/lib/bb/parse/parse_py/BBHandler.py +0 -3
@@ -131,9 +131,6 @@ def handle(fn, d, include): abs_fn = resolve_file(fn, d) - if include: - bb.parse.mark_dependency(d, abs_fn) - # actual loading statements = get_statements(fn, abs_fn, base_name)
bitbake/lib/bb/parse/parse_py/ConfHandler.py +0 -3
@@ -134,9 +134,6 @@ def handle(fn, data, include): abs_fn = resolve_file(fn, data) f = open(abs_fn, 'r') - if include: - bb.parse.mark_dependency(data, abs_fn) - statements = ast.StatementGroup() lineno = 0 while True:
Signature · 1 line
2.7.4
PATCH 7/8 bitbake: cooker: fix event notifications
In reply to Robert YangThere are two kinds of events:
* directory
We always need handle the event since we don't know what inside the directory, for example:
$ bitbake --server-only -T -1
$ bitbake -p
$ mkdir ${TOPDIR}/classes
$ cp /path/to/base.bbclass classesThe "${TOPDIR}/classes" is a new directory, the event only reports that the directory is created, we don't know there is a base.bbclass unless the parser runs.
* file:
Use bb.parse.supports() to check whether bitbake supports it or not.
And now the bbwatchedfiles is not needed anymore.
--- bitbake/lib/bb/cooker.py | 34 ++++++++++++++++++---------------- 1 file changed, 18 insertions(+), 16 deletions(-)
bitbake/lib/bb/cooker.py +18 -16
@@ -177,14 +177,12 @@ class BBCooker: self.configwatcher = pyinotify.WatchManager() self.configwatcher.bbseen = [] - self.configwatcher.bbwatchedfiles = [] self.confignotifier = pyinotify.Notifier(self.configwatcher, self.config_notifications) self.watchmask = pyinotify.IN_CLOSE_WRITE | pyinotify.IN_CREATE | pyinotify.IN_DELETE | \ pyinotify.IN_DELETE_SELF | pyinotify.IN_MODIFY | pyinotify.IN_MOVE_SELF | \ pyinotify.IN_MOVED_FROM | pyinotify.IN_MOVED_TO self.watcher = pyinotify.WatchManager() self.watcher.bbseen = [] - self.watcher.bbwatchedfiles = [] self.notifier = pyinotify.Notifier(self.watcher, self.notifications) # If being called by something like tinfoil, we need to clean cached data
@@ -247,10 +245,15 @@ class BBCooker: self.baseconfig_valid = False bb.parse.clear_cache() return - if not event.pathname in self.configwatcher.bbwatchedfiles: - return - if not event.pathname in self.inotify_modified_files: - self.inotify_modified_files.append(event.pathname) + + # Check whether it is a supported type when it is a file + if not event.dir: + if not bb.parse.supports(event.pathname, data): + return + if not event.pathname in self.inotify_modified_files: + self.inotify_modified_files.append(event.pathname) + + bb.debug(1, "Get config inotify event: %s" % event) self.baseconfig_valid = False def notifications(self, event):
@@ -259,18 +262,21 @@ class BBCooker: self.parsecache_valid = False bb.parse.clear_cache() return - if event.pathname.endswith("bitbake-cookerdaemon.log") \ - or event.pathname.endswith("bitbake.lock"): - return - if not event.pathname in self.inotify_modified_files: - self.inotify_modified_files.append(event.pathname) + + # Check whether it is a supported type when it is a file + if not event.dir: + if not bb.parse.supports(event.pathname, data): + return + if not event.pathname in self.inotify_modified_files: + self.inotify_modified_files.append(event.pathname) + + bb.debug(1, "Get inotify event: %s" % event) self.parsecache_valid = False def add_filewatch(self, deps, watcher=None, dirs=False): if not watcher: watcher = self.watcher for i in deps: - watcher.bbwatchedfiles.append(i[0]) if dirs: f = i[0] else:
@@ -278,19 +284,15 @@ class BBCooker: if f in watcher.bbseen: continue watcher.bbseen.append(f) - watchtarget = None while True: # We try and add watches for files that don't exist but if they did, would influence # the parser. The parent directory of these files may not exist, in which case we need # to watch any parent that does exist for changes. try: watcher.add_watch(f, self.watchmask, quiet=False) - if watchtarget: - watcher.bbwatchedfiles.append(watchtarget) break except pyinotify.WatchManagerError as e: if 'ENOENT' in str(e): - watchtarget = f f = os.path.dirname(f) if f in watcher.bbseen: break
Signature · 1 line
2.7.4
PATCH 8/8 bitbake: cooker: improve inotify handling
In reply to Robert YangEach recipe calls bb_cache.add_info() and it calls add_filewatch() for every depend file (not only existed files, but also non-exsited/attempted ones), which caused the parsing and loading very slow. The watchers are global, so we can only pass new depend files to add_filewatch() to improve the performance.
Here is the testing of 54 layers before and after the patch.
Parsing Loading Before 35s 33s Now 11s 9s
24 seconds are saved for parsing, and 24s are saved for loading.
--- bitbake/lib/bb/cache.py | 9 ++++++--- bitbake/lib/bb/cooker.py | 24 +++++++++++++++--------- 2 files changed, 21 insertions(+), 12 deletions(-)
bitbake/lib/bb/cache.py +6 -3
@@ -713,12 +713,15 @@ class Cache(NoCache): def mtime(cachefile): return bb.parse.cached_mtime_noerror(cachefile) - def add_info(self, filename, info_array, cacheData, parsed=None, watcher=None): + def add_info(self, filename, info_array, cacheData, parsed=None, watcher=None, watch_manager=None): if isinstance(info_array[0], CoreRecipeInfo) and (not info_array[0].skipped): cacheData.add_from_recipeinfo(filename, info_array) - if watcher: - watcher(info_array[0].file_depends) + if watcher and watch_manager: + deps_set = set() + for deps in info_array[0].file_depends: + deps_set.add(deps[0]) + watcher(deps_set - watch_manager.bbseen) if not self.has_cache: return
bitbake/lib/bb/cooker.py +15 -9
@@ -176,13 +176,13 @@ class BBCooker: self.configuration = configuration self.configwatcher = pyinotify.WatchManager() - self.configwatcher.bbseen = [] + self.configwatcher.bbseen = set() self.confignotifier = pyinotify.Notifier(self.configwatcher, self.config_notifications) self.watchmask = pyinotify.IN_CLOSE_WRITE | pyinotify.IN_CREATE | pyinotify.IN_DELETE | \ pyinotify.IN_DELETE_SELF | pyinotify.IN_MODIFY | pyinotify.IN_MOVE_SELF | \ pyinotify.IN_MOVED_FROM | pyinotify.IN_MOVED_TO self.watcher = pyinotify.WatchManager() - self.watcher.bbseen = [] + self.watcher.bbseen = set() self.notifier = pyinotify.Notifier(self.watcher, self.notifications) # If being called by something like tinfoil, we need to clean cached data
@@ -278,12 +278,14 @@ class BBCooker: watcher = self.watcher for i in deps: if dirs: - f = i[0] + f = i else: - f = os.path.dirname(i[0]) + f = os.path.dirname(i) if f in watcher.bbseen: continue - watcher.bbseen.append(f) + # Add both files and directories to to bbseen + watcher.bbseen.add(f) + watcher.bbseen.add(i) while True: # We try and add watches for files that don't exist but if they did, would influence # the parser. The parent directory of these files may not exist, in which case we need
@@ -296,7 +298,7 @@ class BBCooker: f = os.path.dirname(f) if f in watcher.bbseen: break - watcher.bbseen.append(f) + watcher.bbseen.add(f) continue if 'ENOSPC' in str(e): providerlog.error("No space left on device or exceeds fs.inotify.max_user_watches?")
@@ -367,7 +369,10 @@ class BBCooker: self.disableDataTracking() self.data.renameVar("__depends", "__base_depends") - self.add_filewatch(self.data.getVar("__base_depends", False), self.configwatcher) + deps_set = set() + for deps in self.data.getVar("__base_depends", False): + deps_set.add(deps[0]) + self.add_filewatch(deps_set, self.configwatcher) self.baseconfig_valid = True self.parsecache_valid = False
@@ -1510,7 +1515,7 @@ class BBCooker: # Add inotify watches for directories searched for bb/bbappend files for dirent in searchdirs: - self.add_filewatch([[dirent]], dirs=True) + self.add_filewatch([dirent], dirs=True) self.parser = CookerParser(self, filelist, masked) self.parsecache_valid = True
@@ -2144,7 +2149,8 @@ class CookerParser(object): self.cooker.skiplist[virtualfn] = SkippedPackage(info_array[0]) (fn, cls, mc) = bb.cache.virtualfn2realfn(virtualfn) self.bb_cache.add_info(virtualfn, info_array, self.cooker.recipecaches[mc], - parsed=parsed, watcher = self.cooker.add_filewatch) + parsed=parsed, watcher=self.cooker.add_filewatch, + watch_manager=self.cooker.watcher) return True def reparse(self, filename):
Signature · 1 line
2.7.4
Re: [PATCH 7/8] bitbake: cooker: fix event notifications
In reply to Robert YangQuoted 19 lines · Robert Yang
There are two kinds of events:
* directory
We always need handle the event since we don't know what inside the directory, for example:$ bitbake --server-only -T -1 $ bitbake -p $ mkdir ${TOPDIR}/classes $ cp /path/to/base.bbclass classesThe "${TOPDIR}/classes" is a new directory, the event only reports that the
directory is created, we don't know there is a base.bbclass unless the parser
runs.* file:
Usebb.parse.supports()to check whether bitbake supports it or not.
Quoted 6 lines
And now the bbwatchedfiles is not needed anymore.
Signed-off-by Robert Yang <[email protected]>---
bitbake/lib/bb/cooker.py | 34 ++++++++++++++++++----------------1 file changed, 18 insertions(+), 16 deletions(-)
I'm not sure you can do this. For example, bitbake itself doesn't know/care about *.patch files, however if one is referenced in a SRC_URI and ends up on the watch list, would this change break a patch file changing causing task checksums to change?
Its looking like we need better tests for some of this code too :/.
Cheers,
Richard
Re: [PATCH 4/8] bitbake: cooker: don't preserve BB_CONSOLELOG
In reply to Robert YangQuoted 15 lines · Robert Yang
Fixed:
$ bitbake --server-only -T -1 Set MACHINE = "qemux86" in conf/local.conf $ bitbake quilt Set MACHINE = "qemuppc" in conf/local.conf $ bitbake quiltThe log still goes into tmp/log/cooker/qemux86 in the second run, this is
incorrect (should be tmp/log/cooker/qemuppc). I checked the code, the ui
initializes it every time when it starts, so let it use the up-to- date
one rather than old one.
Quoted 4 lines
Signed-off-by Robert Yang <[email protected]>---
bitbake/lib/bb/cooker.py | 8 --------1 file changed, 8 deletions(-)
This effectively reverts:
Sadly I didn't put a bug number in that commit message but I do remember there being a good reason to do this.
I think the problem is a "UI" can run multiple commands with reset events within that command stream and it would only have one console log. I also believe there may have been issues where toaster would have been unable to find the logs if they kept moving around.
The console log handling probably needs more thought as there are clearly multiple issues here. We may want to move it out of a directory which is machine specific?
Cheers,
Richard
Quoted 29 lines
bitbake/lib/bb/cooker.py +0
-0@@ -326,11 +326,6 @@ class BBCooker:self.state = state.initial
self.caches_array = []
- # Need to preserveBB_CONSOLELOGover resets
- consolelog = None
- ifhasattr(self, "data"):
- consolelog =self.data.getVar("BB_CONSOLELOG")-
if CookerFeatures.BASEDATASTORE_TRACKING in self.featureset:
self.enableDataTracking()bitbake/lib/bb/cooker.py +0
-0@@ -358,9 +353,6 @@ class BBCooker:self.data_hash = self.databuilder.data_hash self.extraconfigdata = {}
- if consolelog:
- self.data.setVar("BB_CONSOLELOG", consolelog) -
self.data.setVar('BB_CMDLINE', self.ui_cmdline)
#
--
2.7.4
Re: [PATCH 7/8] bitbake: cooker: fix event notifications
In reply to Richard PurdieHi RP,
Quoted 27 lines · Richard Purdie
Quoted 19 lines · Robert Yang
There are two kinds of events:
* directory
We always need handle the event since we don't know what inside the directory, for example:$ bitbake --server-only -T -1 $ bitbake -p $ mkdir ${TOPDIR}/classes $ cp /path/to/base.bbclass classesThe "${TOPDIR}/classes" is a new directory, the event only reports
that the
directory is created, we don't know there is a base.bbclass unless
the parser
runs.* file:
Usebb.parse.supports()to check whether bitbake supports it or
not.Quoted 6 lines
And now the bbwatchedfiles is not needed anymore.
Signed-off-by Robert Yang <[email protected]>--- bitbake/lib/bb/cooker.py | 34 ++++++++++++++++++---------------- 1 file changed, 18 insertions(+), 16 deletions(-)
Quoted 4 lines
I'm not sure you can do this. For example, bitbake itself doesn't know/care about *.patch files, however if one is referenced in a
SRC_URIand ends up on the watch list, would this change break a patch file changing causing task checksums to change?
I can confirm the the following 2 things:
1) The watcher only watches __depends (and __base_depends) which only contains files supported by bb.parse.supports(), so use bb.parse.supports() to check it is OK (or maybe it is the best way).
2) The *.patch is not part of of __depends, but file_checksums, and they are not watched by watchers. When a patch is updated, it would compare the saved mtime and current mtime (in lib/bb/checksum.py), and decide whether need recompute.
So I think that this patch is fine.
The testcase:
$ bitbake --server-only -T -1 $ bitbake quilt-native
# Update quilt-native's patch to test the rebuild:
$ echo "hello world" >> /path/to/meta/recipes-devtools/quilt/quilt/0001-tests-Allow-different-output-from-mv.patch
# Build quilt-native again
$ bitbake quilt-native
NOTE: Tasks Summary: Attempted 10 tasks of which 0 didn't need to be rerun and all succeeded.
We can see that it has been rebuilt.
// Robert
Quoted 8 lines
Its looking like we need better tests for some of this code too :/.
Cheers,
Richard
Re: [PATCH 4/8] bitbake: cooker: don't preserve BB_CONSOLELOG
In reply to Richard PurdieHi RP,
Quoted 25 lines · Richard Purdie
Quoted 15 lines · Robert Yang
Fixed:
$ bitbake --server-only -T -1 Set MACHINE = "qemux86" in conf/local.conf $ bitbake quilt Set MACHINE = "qemuppc" in conf/local.conf $ bitbake quiltThe log still goes into tmp/log/cooker/qemux86 in the second run, this is
incorrect (should be tmp/log/cooker/qemuppc). I checked the code, the ui
initializes it every time when it starts, so let it use the up-to- date
one rather than old one.Quoted 4 lines
Signed-off-by Robert Yang <[email protected]>--- bitbake/lib/bb/cooker.py | 8 -------- 1 file changed, 8 deletions(-)This effectively reverts:
Quoted 7 lines
Sadly I didn't put a bug number in that commit message but I do remember there being a good reason to do this.
I think the problem is a "UI" can run multiple commands with reset events within that command stream and it would only have one console log. I also believe there may have been issues where toaster would have been unable to find the logs if they kept moving around.
I've found the bug:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8411
I think that it has been fixed by:
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8373
http://git.yoctoproject.org/cgit/cgit.cgi/poky/commit/?id=a0791c19db7d5b33ce7a4e3614a574ad4a319a84
I've tried the following commands with my patch:
$ . /workspace1/lyang1/poky/bitbake/bin/toaster start noweb $ MACHINE=qemuppc bitbake quilt-native
The log is tmp/log/cooker/qemuppc/build_20180208_120722.942.log
$ MACHINE=qemux86 bitbake quilt-native
The log is tmp/log/cooker/qemux86/build_20180208_121605.382.log
And the log contents are correct.
So I think that we don't need preserve BB_CONSOLELOG anymore.
// Robert
Quoted 39 lines
The console log handling probably needs more thought as there are clearly multiple issues here. We may want to move it out of a directory which is machine specific?
Cheers,
Richard
Quoted 29 lines
bitbake/lib/bb/cooker.py +0
-8@@ -326,11 +326,6 @@ class BBCooker: self.state = state.initial self.caches_array = [] - # Need to preserve BB_CONSOLELOG over resets - consolelog = None - if hasattr(self, "data"): - consolelog = self.data.getVar("BB_CONSOLELOG") - if CookerFeatures.BASEDATASTORE_TRACKING in self.featureset: self.enableDataTracking()@@ -358,9 +353,6 @@ class BBCooker: self.data_hash = self.databuilder.data_hash self.extraconfigdata = {} - if consolelog: - self.data.setVar("BB_CONSOLELOG", consolelog) - self.data.setVar('BB_CMDLINE', self.ui_cmdline) #Signature · 1 line
2.7.4