Add some missing buildman features and deprecate MAKEALL
v9 cover Add some missing buildman features and deprecate MAKEALL
Buildman has been around for a little over a year and is used by a fair number of U-Boot developers. However quite a few people still use MAKEALL.
Buildman was intended to replace MAKEALL, so perhaps now is a good time to start that process.
The reasons to deprecate MAKEALL are:
- We don't want to maintain two build systems
- Buildman is typically faster
- Buildman has a lot more features
This series adds a few features to buildman to fill some gaps, adds some information into the README on how to migrate from MAKEALL, and adds a deprecation message to MAKEALL.
Changes in v9:
- Add new patch to avoid changing the order of tags
- Add new patch to set the return code to indicate build result
- Add new patch to allow make-flags variables to include '-' and '_'
- Add new patch to implement --exclude option
- Add new patch to remove the directory prefix from each error line
- Add new patch to support showing which boards caused which errors
- Add new patch to separate out display of warnings and errors
- Add new patch to fix detection of git version
Changes in v8:
- Add new patch to disable the pager in git
Changes in v7:
- Add new patch to fix the 'reverse' bug
- Remove already-applied patches from the series
- Add the deprecation message at the end of the build also
- Drop the 'colour' patch sadly
Changes in v6:
- Add new patch to fix indentation in teminal.py
- Add new patch to fix patman unit tests
- Add new patch to remove patman's -a option
Changes in v5:
- Drop patch to search for *cc instead of *gcc for the compiler
Simon Glass (14):
patman: Support the 'reverse' option for 'git log'
patman: Fix indentation in terminal.py
patman: Correct unit tests to run correctly
patman: Remove the -a option
patman: Use --no-pager' to stop git from forking a pager
patman: Avoid changing the order of tags
buildman: Set the return code to indicate build result
buildman: Allow make-flags variables to include '-' and '_'
buildman: Implement an option to exclude boards from the build
buildman: Remove the directory prefix from each error line
buildman: Add an option to show which boards caused which errors
buildman: Separate out display of warnings and errors
patman: Fix detection of git version
RFC: Deprecate MAKEALL
MAKEALL | 10 +++ tools/buildman/README | 27 ++++++-- tools/buildman/board.py | 31 +++++++--- tools/buildman/builder.py | 133 ++++++++++++++++++++++++++++++++++------ tools/buildman/builderthread.py | 22 ++++++- tools/buildman/buildman.py | 8 ++- tools/buildman/control.py | 20 ++++-- tools/buildman/toolchain.py | 2 +- tools/patman/gitutil.py | 102 +++--------------------------- tools/patman/patchstream.py | 21 ++----- tools/patman/patman.py | 7 --- tools/patman/terminal.py | 112 +++++++++++++++++---------------- tools/patman/test.py | 13 ++-- 13 files changed, 298 insertions(+), 210 deletions(-)
Signature · 1 line
2.1.0.rc2.206.gedb03e5
PATCH v9 1/14 patman: Support the 'reverse' option for 'git log'
This option is currently not supported, but needs to be, for buildman to operate as expected.
---
Changes in v9: None
Changes in v8: None
Changes in v7:
- Add new patch to fix the 'reverse' bug
Changes in v6: None
Changes in v5: None
tools/patman/gitutil.py | 2 ++ 1 file changed, 2 insertions(+)
tools/patman/gitutil.py +2 -0
@@ -38,6 +38,8 @@ def LogCmd(commit_range, git_dir=None, oneline=False, reverse=False, cmd.append('--oneline') if use_no_decorate: cmd.append('--no-decorate') + if reverse: + cmd.append('--reverse') if count is not None: cmd.append('-n%d' % count) if commit_range:
Signature · 1 line
2.1.0.rc2.206.gedb03e5
PATCH v9 2/14 patman: Fix indentation in terminal.py
In reply to Simon GlassThis code came from a different project with 2-character indentation. Fix it for U-Boot.
---
Changes in v9: None
Changes in v8: None
Changes in v7: None
Changes in v6:
- Add new patch to fix indentation in teminal.py
Changes in v5: None
tools/patman/terminal.py | 108 ++++++++++++++++++++++++----------------------- 1 file changed, 55 insertions(+), 53 deletions(-)
tools/patman/terminal.py +55 -53
@@ -15,66 +15,68 @@ import sys COLOR_IF_TERMINAL, COLOR_ALWAYS, COLOR_NEVER = range(3) class Color(object): - """Conditionally wraps text in ANSI color escape sequences.""" - BLACK, RED, GREEN, YELLOW, BLUE, MAGENTA, CYAN, WHITE = range(8) - BOLD = -1 - BRIGHT_START = '\033[1;%dm' - NORMAL_START = '\033[22;%dm' - BOLD_START = '\033[1m' - RESET = '\033[0m' + """Conditionally wraps text in ANSI color escape sequences.""" + BLACK, RED, GREEN, YELLOW, BLUE, MAGENTA, CYAN, WHITE = range(8) + BOLD = -1 + BRIGHT_START = '\033[1;%dm' + NORMAL_START = '\033[22;%dm' + BOLD_START = '\033[1m' + RESET = '\033[0m' - def __init__(self, colored=COLOR_IF_TERMINAL): - """Create a new Color object, optionally disabling color output. + def __init__(self, colored=COLOR_IF_TERMINAL): + """Create a new Color object, optionally disabling color output. - Args: - enabled: True if color output should be enabled. If False then this - class will not add color codes at all. - """ - self._enabled = (colored == COLOR_ALWAYS or - (colored == COLOR_IF_TERMINAL and os.isatty(sys.stdout.fileno()))) + Args: + enabled: True if color output should be enabled. If False then this + class will not add color codes at all. + """ + self._enabled = (colored == COLOR_ALWAYS or + (colored == COLOR_IF_TERMINAL and os.isatty(sys.stdout.fileno()))) - def Start(self, color, bright=True): - """Returns a start color code. + def Start(self, color, bright=True): + """Returns a start color code. - Args: - color: Color to use, .e.g BLACK, RED, etc. + Args: + color: Color to use, .e.g BLACK, RED, etc. - Returns: - If color is enabled, returns an ANSI sequence to start the given color, - otherwise returns empty string - """ - if self._enabled: - base = self.BRIGHT_START if bright else self.NORMAL_START - return base % (color + 30) - return '' + Returns: + If color is enabled, returns an ANSI sequence to start the given + color, otherwise returns empty string + """ + if self._enabled: + base = self.BRIGHT_START if bright else self.NORMAL_START + return base % (color + 30) + return '' - def Stop(self): - """Retruns a stop color code. + def Stop(self): + """Retruns a stop color code. - Returns: - If color is enabled, returns an ANSI color reset sequence, otherwise - returns empty string - """ - if self._enabled: - return self.RESET - return '' + Returns: + If color is enabled, returns an ANSI color reset sequence, + otherwise returns empty string + """ + if self._enabled: + return self.RESET + return '' - def Color(self, color, text, bright=True): - """Returns text with conditionally added color escape sequences. + def Color(self, color, text, bright=True): + """Returns text with conditionally added color escape sequences. - Keyword arguments: - color: Text color -- one of the color constants defined in this class. - text: The text to color. + Keyword arguments: + color: Text color -- one of the color constants defined in this + class. + text: The text to color. - Returns: - If self._enabled is False, returns the original text. If it's True, - returns text with color escape sequences based on the value of color. - """ - if not self._enabled: - return text - if color == self.BOLD: - start = self.BOLD_START - else: - base = self.BRIGHT_START if bright else self.NORMAL_START - start = base % (color + 30) - return start + text + self.RESET + Returns: + If self._enabled is False, returns the original text. If it's True, + returns text with color escape sequences based on the value of + color. + """ + if not self._enabled: + return text + if color == self.BOLD: + start = self.BOLD_START + else: + base = self.BRIGHT_START if bright else self.NORMAL_START + start = base % (color + 30) + return start + text + self.RESET
Signature · 1 line
2.1.0.rc2.206.gedb03e5
PATCH v9 3/14 patman: Correct unit tests to run correctly
In reply to Simon GlassIt seems that doctest behaves differently now, and some of the unit tests do not run. Adjust the tests to work correctly.
./tools/patman/patman --test
<unittest.result.TestResult run=10 errors=0 failures=0>
---
Changes in v9: None
Changes in v8: None
Changes in v7: None
Changes in v6:
- Add new patch to fix patman unit tests
Changes in v5: None
tools/patman/gitutil.py | 8 ++++---- tools/patman/patchstream.py | 7 +++++-- tools/patman/terminal.py | 8 ++++++-- tools/patman/test.py | 13 +++++++------ 4 files changed, 22 insertions(+), 14 deletions(-)
tools/patman/gitutil.py +4 -4
@@ -478,13 +478,13 @@ def LookupEmail(lookup_name, alias=None, raise_on_error=True, level=0): ... OSError: Recursive email alias at 'other' >>> LookupEmail('odd', alias, raise_on_error=False) - \033[1;31mAlias 'odd' not found\033[0m + Alias 'odd' not found [] >>> # In this case the loop part will effectively be ignored. >>> LookupEmail('loop', alias, raise_on_error=False) - \033[1;31mRecursive email alias at 'other'\033[0m - \033[1;31mRecursive email alias at 'john'\033[0m - \033[1;31mRecursive email alias at 'mary'\033[0m + Recursive email alias at 'other' + Recursive email alias at 'john' + Recursive email alias at 'mary' ['j.bloggs at napier.co.nz', 'm.poppins at cloud.net'] """ if not alias:
tools/patman/patchstream.py +5 -2
@@ -275,7 +275,7 @@ class PatchStream: # Suppress duplicate signoffs elif signoff_match: - if (self.is_log or + if (self.is_log or not self.commit or self.commit.CheckDuplicateSignoff(signoff_match.group(1))): out = [line]
@@ -312,7 +312,10 @@ class PatchStream: out = [] log = self.series.MakeChangeLog(self.commit) out += self.FormatTags(self.tags) - out += [line] + self.commit.notes + [''] + log + out += [line] + if self.commit: + out += self.commit.notes + out += [''] + log elif self.found_test: if not re_allowed_after_test.match(line): self.lines_after_test += 1
tools/patman/terminal.py +6 -2
@@ -30,8 +30,12 @@ class Color(object): enabled: True if color output should be enabled. If False then this class will not add color codes at all. """ - self._enabled = (colored == COLOR_ALWAYS or - (colored == COLOR_IF_TERMINAL and os.isatty(sys.stdout.fileno()))) + try: + self._enabled = (colored == COLOR_ALWAYS or + (colored == COLOR_IF_TERMINAL and + os.isatty(sys.stdout.fileno()))) + except: + self._enabled = False def Start(self, color, bright=True): """Returns a start color code.
tools/patman/test.py +7 -6
@@ -55,6 +55,7 @@ This adds functions to enable/disable clocks and reset to on-chip peripherals. Signed-off-by: Simon Glass <[email protected]> --- + arch/arm/cpu/armv7/tegra2/Makefile | 2 +- arch/arm/cpu/armv7/tegra2/ap20.c | 57 ++---- arch/arm/cpu/armv7/tegra2/clock.c | 163 +++++++++++++++++
@@ -200,7 +201,7 @@ index 0000000..2234c87 self.assertEqual(result.errors, 0) self.assertEqual(result.warnings, 0) self.assertEqual(result.checks, 0) - self.assertEqual(result.lines, 67) + self.assertEqual(result.lines, 56) os.remove(inf) def testNoSignoff(self):
@@ -211,18 +212,18 @@ index 0000000..2234c87 self.assertEqual(result.errors, 1) self.assertEqual(result.warnings, 0) self.assertEqual(result.checks, 0) - self.assertEqual(result.lines, 67) + self.assertEqual(result.lines, 56) os.remove(inf) def testSpaces(self): inf = self.SetupData('spaces') result = checkpatch.CheckPatch(inf) self.assertEqual(result.ok, False) - self.assertEqual(len(result.problems), 1) + self.assertEqual(len(result.problems), 2) self.assertEqual(result.errors, 0) - self.assertEqual(result.warnings, 1) + self.assertEqual(result.warnings, 2) self.assertEqual(result.checks, 0) - self.assertEqual(result.lines, 67) + self.assertEqual(result.lines, 56) os.remove(inf) def testIndent(self):
@@ -233,7 +234,7 @@ index 0000000..2234c87 self.assertEqual(result.errors, 0) self.assertEqual(result.warnings, 0) self.assertEqual(result.checks, 1) - self.assertEqual(result.lines, 67) + self.assertEqual(result.lines, 56) os.remove(inf)
Signature · 1 line
2.1.0.rc2.206.gedb03e5
PATCH v9 4/14 patman: Remove the -a option
In reply to Simon GlassIt seems that this is no longer needed, since checkpatch.pl will catch whitespace problems in patches. Also the option is not widely used, so it seems safe to just remove it.
---
Changes in v9: None
Changes in v8: None
Changes in v7: None
Changes in v6:
- Add new patch to remove patman's -a option
Changes in v5: None
tools/patman/gitutil.py | 88 ------------------------------------------------- tools/patman/patman.py | 7 ---- 2 files changed, 95 deletions(-)
tools/patman/gitutil.py +0 -88
@@ -215,94 +215,6 @@ def CreatePatches(start, count, series): else: return None, files -def ApplyPatch(verbose, fname): - """Apply a patch with git am to test it - - TODO: Convert these to use command, with stderr option - - Args: - fname: filename of patch file to apply - """ - col = terminal.Color() - cmd = ['git', 'am', fname] - pipe = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - stdout, stderr = pipe.communicate() - re_error = re.compile('^error: patch failed: (.+):(\d+)') - for line in stderr.splitlines(): - if verbose: - print line - match = re_error.match(line) - if match: - print checkpatch.GetWarningMsg(col, 'warning', match.group(1), - int(match.group(2)), 'Patch failed') - return pipe.returncode == 0, stdout - -def ApplyPatches(verbose, args, start_point): - """Apply the patches with git am to make sure all is well - - Args: - verbose: Print out 'git am' output verbatim - args: List of patch files to apply - start_point: Number of commits back from HEAD to start applying. - Normally this is len(args), but it can be larger if a start - offset was given. - """ - error_count = 0 - col = terminal.Color() - - # Figure out our current position - cmd = ['git', 'name-rev', 'HEAD', '--name-only'] - pipe = subprocess.Popen(cmd, stdout=subprocess.PIPE) - stdout, stderr = pipe.communicate() - if pipe.returncode: - str = 'Could not find current commit name' - print col.Color(col.RED, str) - print stdout - return False - old_head = stdout.splitlines()[0] - if old_head == 'undefined': - str = "Invalid HEAD '%s'" % stdout.strip() - print col.Color(col.RED, str) - return False - - # Checkout the required start point - cmd = ['git', 'checkout', 'HEAD~%d' % start_point] - pipe = subprocess.Popen(cmd, stdout=subprocess.PIPE, - stderr=subprocess.PIPE) - stdout, stderr = pipe.communicate() - if pipe.returncode: - str = 'Could not move to commit before patch series' - print col.Color(col.RED, str) - print stdout, stderr - return False - - # Apply all the patches - for fname in args: - ok, stdout = ApplyPatch(verbose, fname) - if not ok: - print col.Color(col.RED, 'git am returned errors for %s: will ' - 'skip this patch' % fname) - if verbose: - print stdout - error_count += 1 - cmd = ['git', 'am', '--skip'] - pipe = subprocess.Popen(cmd, stdout=subprocess.PIPE) - stdout, stderr = pipe.communicate() - if pipe.returncode != 0: - print col.Color(col.RED, 'Unable to skip patch! Aborting...') - print stdout - break - - # Return to our previous position - cmd = ['git', 'checkout', old_head] - pipe = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) - stdout, stderr = pipe.communicate() - if pipe.returncode: - print col.Color(col.RED, 'Could not move back to head commit') - print stdout, stderr - return error_count == 0 - def BuildEmailList(in_list, tag=None, alias=None, raise_on_error=True): """Build a list of email addresses based on an input list.
tools/patman/patman.py +0 -7
@@ -25,9 +25,6 @@ import test parser = OptionParser() -parser.add_option('-a', '--no-apply', action='store_false', - dest='apply_patches', default=True, - help="Don't test-apply patches with git am") parser.add_option('-H', '--full-help', action='store_true', dest='full_help', default=False, help='Display the README file') parser.add_option('-c', '--count', dest='count', type='int',
@@ -143,10 +140,6 @@ else: ok = checkpatch.CheckPatches(options.verbose, args) else: ok = True - if options.apply_patches: - if not gitutil.ApplyPatches(options.verbose, args, - options.count + options.start): - ok = False cc_file = series.MakeCcFile(options.process_tags, cover_fname, not options.ignore_bad_tags)
Signature · 1 line
2.1.0.rc2.206.gedb03e5
PATCH v9 5/14 patman: Use --no-pager' to stop git from forking a pager
In reply to Simon GlassIn a headless environment the pager can apparently hang. We don't want a pager anyway so let's request that none be used.
---
Changes in v9: None
Changes in v8:
- Add new patch to disable the pager in git
Changes in v7: None
Changes in v6: None
Changes in v5: None
tools/patman/gitutil.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/patman/gitutil.py +1 -1
@@ -33,7 +33,7 @@ def LogCmd(commit_range, git_dir=None, oneline=False, reverse=False, cmd = ['git'] if git_dir: cmd += ['--git-dir', git_dir] - cmd += ['log', '--no-color'] + cmd += ['--no-pager', 'log', '--no-color'] if oneline: cmd.append('--oneline') if use_no_decorate:
Signature · 1 line
2.1.0.rc2.206.gedb03e5
PATCH v9 6/14 patman: Avoid changing the order of tags
In reply to Simon Glasspatman collects tags that it sees in the commit and places them nicely sorted at the end of the patch. However, this is not really necessary and in fact is apparently not desirable.
---
Changes in v9:
- Add new patch to avoid changing the order of tags
Changes in v8: None
Changes in v7: None
Changes in v6: None
Changes in v5: None
tools/patman/patchstream.py | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-)
tools/patman/patchstream.py +1 -13
@@ -72,7 +72,6 @@ class PatchStream: self.in_change = 0 # Non-zero if we are in a change list self.blank_count = 0 # Number of blank lines stored up self.state = STATE_MSG_HEADER # What state are we in? - self.tags = [] # Tags collected, like Tested-by... self.signoff = [] # Contents of signoff line self.commit = None # Current commit
@@ -113,16 +112,6 @@ class PatchStream: self.series.AddCommit(self.commit) self.commit = None - def FormatTags(self, tags): - out_list = [] - for tag in sorted(tags): - if tag.startswith('Cc:'): - tag_list = tag[4:].split(',') - out_list += gitutil.BuildEmailList(tag_list, 'Cc:') - else: - out_list.append(tag) - return out_list - def ProcessLine(self, line): """Process a single line of a patch file or commit log
@@ -271,7 +260,7 @@ class PatchStream: elif tag_match.group(1) == 'Patch-cc': self.commit.AddCc(tag_match.group(2).split(',')) else: - self.tags.append(line); + out = [line] # Suppress duplicate signoffs elif signoff_match:
@@ -311,7 +300,6 @@ class PatchStream: # Output the tags (signeoff first), then change list out = [] log = self.series.MakeChangeLog(self.commit) - out += self.FormatTags(self.tags) out += [line] if self.commit: out += self.commit.notes
Signature · 1 line
2.1.0.rc2.206.gedb03e5
PATCH v9 7/14 buildman: Set the return code to indicate build result
In reply to Simon GlassWhen buildman finds errors/warnings when building, set the return code to indicate this.
---
Changes in v9:
- Add new patch to set the return code to indicate build result
Changes in v8: None
Changes in v7: None
Changes in v6: None
Changes in v5: None
tools/buildman/README | 6 ++++++ tools/buildman/builder.py | 5 +++++ tools/buildman/buildman.py | 3 ++- tools/buildman/control.py | 9 +++++++-- 4 files changed, 20 insertions(+), 3 deletions(-)
tools/buildman/README +6 -0
@@ -690,6 +690,12 @@ Other options Buildman has various other command line options. Try --help to see them. +When doing builds, Buildman's return code will reflect the overall result: + + 0 (success) No errors or warnings found + 128 Errors found + 129 Warnings found + How to change from MAKEALL ==========================
tools/buildman/builder.py +5 -0
@@ -1031,6 +1031,10 @@ class Builder: value is Board object keep_outputs: True to save build output files verbose: Display build results as they are completed + Returns: + Tuple containing: + - number of boards that failed to build + - number of boards that issued warnings """ self.commit_count = len(commits) if commits else 1 self.commits = commits
@@ -1060,3 +1064,4 @@ class Builder: self.out_queue.join() print self.ClearLine(0) + return (self.fail, self.warned)
tools/buildman/buildman.py +2 -1
@@ -136,4 +136,5 @@ elif options.full_help: # Build selected commits for selected boards else: - control.DoBuildman(options, args) + ret_code = control.DoBuildman(options, args) + sys.exit(ret_code)
tools/buildman/control.py +7 -2
@@ -94,7 +94,7 @@ def DoBuildman(options, args): if options.list_tool_chains: toolchains.List() print - return + return 0 # Work out how many commits to build. We want to build everything on the # branch. We also build the upstream commit as a control so we can see
@@ -219,5 +219,10 @@ def DoBuildman(options, args): options.show_detail = True builder.ShowSummary(commits, board_selected) else: - builder.BuildBoards(commits, board_selected, + fail, warned = builder.BuildBoards(commits, board_selected, options.keep_outputs, options.verbose) + if fail: + return 128 + elif warned: + return 129 + return 0
Signature · 1 line
2.1.0.rc2.206.gedb03e5
PATCH v9 8/14 buildman: Allow make-flags variables to include '-' and '_'
In reply to Simon GlassThese characters are commonly used in variables, so permit them. Also document the permitted characters.
---
Changes in v9:
- Add new patch to allow make-flags variables to include '-' and '_'
Changes in v8: None
Changes in v7: None
Changes in v6: None
Changes in v5: None
tools/buildman/README | 4 +++- tools/buildman/toolchain.py | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-)
tools/buildman/README +3 -1
@@ -670,7 +670,9 @@ snapper9g45=${at91-boards} BUILD_TAG=443 This will use 'make ENABLE_AT91_TEST=1 BUILD_TAG=442' for snapper9260 and 'make ENABLE_AT91_TEST=1 BUILD_TAG=443' for snapper9g45. A special variable ${target} is available to access the target name (snapper9260 and -snapper9g20 in this case). Variables are resolved recursively. +snapper9g20 in this case). Variables are resolved recursively. Note that +variables can only contain the characters A-Z, a-z, 0-9, hyphen (-) and +underscore (_). It is expected that any variables added are dealt with in U-Boot's config.mk file and documented in the README.
tools/buildman/toolchain.py +1 -1
@@ -198,7 +198,7 @@ class Toolchains: >>> tcs.ResolveReferences(var_dict, 'this=${oblique}_set${first}nd') 'this=OBLIQUE_setfi2ndrstnd' """ - re_var = re.compile('(\$\{[a-z0-9A-Z]{1,}\})') + re_var = re.compile('(\$\{[-_a-z0-9A-Z]{1,}\})') while True: m = re_var.search(args)
Signature · 1 line
2.1.0.rc2.206.gedb03e5
PATCH v9 9/14 buildman: Implement an option to exclude boards from the build
In reply to Simon GlassSome boards are known to be broken and it is convenient to be able to exclude them from the build.
Add an --exclude option to specific boards to exclude. This uses the same matching rules as the normal 'include' arguments, and is a comma- separated list of regular expressions.
---
Changes in v9:
- Add new patch to implement --exclude option
Changes in v8: None
Changes in v7: None
Changes in v6: None
Changes in v5: None
tools/buildman/README | 7 +++++++ tools/buildman/board.py | 31 ++++++++++++++++++++++++------- tools/buildman/buildman.py | 3 +++ tools/buildman/control.py | 8 +++++++- 4 files changed, 41 insertions(+), 8 deletions(-)
tools/buildman/README +7 -0
@@ -114,6 +114,13 @@ the '&' operator to limit the selection: * 'freescale & arm sandbox' All Freescale boards with ARM architecture, plus sandbox +You can also use -x to specifically exclude some boards. For example: + + buildmand arm -x nvidia,freescale,.*ball$ + +means to build all arm boards except nvidia, freescale and anything ending +with 'ball'. + It is convenient to use the -n option to see whaat will be built based on the subset given.
tools/buildman/board.py +24 -7
@@ -239,13 +239,14 @@ class Boards: terms.append(term) return terms - def SelectBoards(self, args): + def SelectBoards(self, args, exclude=[]): """Mark boards selected based on args Args: - List of strings specifying boards to include, either named, or - by their target, architecture, cpu, vendor or soc. If empty, all - boards are selected. + args: List of strings specifying boards to include, either named, + or by their target, architecture, cpu, vendor or soc. If + empty, all boards are selected. + exclude: List of boards to exclude, regardless of 'args' Returns: Dictionary which holds the number of boards which were selected
@@ -258,17 +259,33 @@ class Boards: for term in terms: result[str(term)] = 0 + exclude_list = [] + for expr in exclude: + exclude_list.append(Expr(expr)) + for board in self._boards: + matching_term = None + build_it = False if terms: match = False for term in terms: if term.Matches(board.props): - board.build_it = True - result[str(term)] += 1 - result['all'] += 1 + matching_term = str(term) + build_it = True break else: + build_it = True + + # Check that it is not specifically excluded + for expr in exclude_list: + if expr.Matches(board.props): + build_it = False + break + + if build_it: board.build_it = True + if matching_term: + result[matching_term] += 1 result['all'] += 1 return result
tools/buildman/buildman.py +3 -0
@@ -117,6 +117,9 @@ parser.add_option('-u', '--show_unknown', action='store_true', default=False, help='Show boards with unknown build result') parser.add_option('-v', '--verbose', action='store_true', default=False, help='Show build results while the build progresses') +parser.add_option('-x', '--exclude', dest='exclude', + type='string', action='append', + help='Specify a list of boards to exclude, separated by comma') parser.usage += """
tools/buildman/control.py +7 -1
@@ -129,7 +129,13 @@ def DoBuildman(options, args): boards = board.Boards() boards.ReadBoards(os.path.join(options.git, 'boards.cfg')) - why_selected = boards.SelectBoards(args) + + exclude = [] + if options.exclude: + for arg in options.exclude: + exclude += arg.split(',') + + why_selected = boards.SelectBoards(args, exclude) selected = boards.GetSelected() if not len(selected): sys.exit(col.Color(col.RED, 'No matching boards found'))
Signature · 1 line
2.1.0.rc2.206.gedb03e5
PATCH v9 10/14 buildman: Remove the directory prefix from each error line
In reply to Simon GlassThe full path is long and also includes buildman private directories. Clean this up, so that only a relative U-Boot path is shown.
This will change warnings like these:
/home/sjg/c/src/third_party/u-boot/buildman5/.bm-work/00/arch/sandbox/cpu/cpu.c: In function 'timer_get_us':/home/sjg/c/src/third_party/u-boot/buildman5/.bm-work/00/arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable]
/home/sjg/c/src/third_party/u-boot/files/arch/sandbox/cpu/cpu.c: In function 'timer_get_us':/home/sjg/c/src/third_party/u-boot/files/arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable]
to:
arch/sandbox/cpu/cpu.c: In function 'timer_get_us':arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable]
---
Changes in v9:
- Add new patch to remove the directory prefix from each error line
Changes in v8: None
Changes in v7: None
Changes in v6: None
Changes in v5: None
tools/buildman/builderthread.py | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-)
tools/buildman/builderthread.py +21 -1
@@ -103,6 +103,23 @@ class BuilderThread(threading.Thread): return self.builder.do_make(commit, brd, stage, cwd, *args, **kwargs) + def _FilterOutDir(self, text, dirname): + """Filter out a directory name from each line of a block of text + + Args: + text: Text to process (can be multiple lines separated by \n) + dirname: Directory name to remove from the start of each text + line + Returns: + Filtered text. + """ + out = [] + for line in text.split('\n'): + if line.startswith(dirname): + line = line[len(dirname):] + out.append(line) + return '\n'.join(out) + def RunCommit(self, commit_upto, brd, work_dir, do_config, force_build, force_build_failures): """Build a particular commit.
@@ -177,6 +194,7 @@ class BuilderThread(threading.Thread): Mkdir(out_dir) args = [] cwd = work_dir + src_dir = os.path.realpath(work_dir) if not self.builder.in_tree: if commit_upto is None: # In this case we are building in the original source
@@ -189,6 +207,7 @@ class BuilderThread(threading.Thread): work_dir = os.path.realpath(work_dir) args.append('O=%s/build' % work_dir) cwd = None + src_dir = os.getcwd() else: args.append('O=build') args.append('-s')
@@ -209,7 +228,8 @@ class BuilderThread(threading.Thread): if result.return_code == 0: result = self.Make(commit, brd, 'build', cwd, *args, env=env) - result.stdout = config_out + result.stdout + result.stderr = self._FilterOutDir(result.stderr, + src_dir + '/') else: result.return_code = 1 result.stderr = 'No tool chain for %s\n' % brd.arch
Signature · 1 line
2.1.0.rc2.206.gedb03e5
PATCH v9 10/14 buildman: Remove the directory prefix from each error line
Quoted 2 lines · Simon Glass
The full path is long and also includes buildman private directories. Clean this up, so that only a relative U-Boot path is shown.
This is missing a global somewhere perhaps? I do buildman ... -o /tmp/username/toolchain/ and see things like:
"In file included from /tmp/username/toolchain/master/.bm-work/00/..path in u-boot"
But other things are corrected.
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140826/55bd647f/attachment.pgp>
PATCH v9 10/14 buildman: Remove the directory prefix from each error line
Hi Tom,
Quoted 11 lines · Tom Rini
Quoted 2 lines · Simon Glass
The full path is long and also includes buildman private directories. Clean this up, so that only a relative U-Boot path is shown.
This is missing a global somewhere perhaps? I do buildman ... -o
/tmp/username/toolchain/and see things like:
"In file included from/tmp/username/toolchain/master/.bm-work/00/..pathin u-boot"But other things are corrected.
Sorry, you've hit a case that isn't covered. I'll see if I can tweak it.
Regards,
Simon
PATCH v9 11/14 buildman: Add an option to show which boards caused which errors
In reply to Simon GlassAdd a -l option to display a list of offending boards against each error/warning line. The information will be shown in brackets as below:
02: wip
sandbox: + sandbox
arm: + seaboard
+(sandbox) arch/sandbox/cpu/cpu.c: In function 'timer_get_us':
+(sandbox) arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable] +(seaboard) board/nvidia/seaboard/seaboard.c: In function 'pin_mux_mmc':
+(seaboard) board/nvidia/seaboard/seaboard.c:36:9: warning: unused variable 'fred' [-Wunused-variable]
+(seaboard) int fred;
+(seaboard) ^
---
Changes in v9:
- Add new patch to support showing which boards caused which errors
Changes in v8: None
Changes in v7: None
Changes in v6: None
Changes in v5: None
tools/buildman/README | 7 ++++--- tools/buildman/builder.py | 51 ++++++++++++++++++++++++++++++++++++++-------- tools/buildman/buildman.py | 2 ++ tools/buildman/control.py | 3 ++- 4 files changed, 50 insertions(+), 13 deletions(-)
tools/buildman/README +4 -3
@@ -442,7 +442,8 @@ is fixed, but there is a new one at line 126. This is probably only because we added some code and moved the broken line father down the file. If many boards have the same error, then -e will display the error only -once. This makes the output as concise as possible. +once. This makes the output as concise as possible. To see which boards have +each error, use -l. The full build output in this case is available in:
@@ -745,10 +746,10 @@ followed by (afterwards, or perhaps concurrently in another terminal): to see the results of the build. Rather than showing you all the output, buildman just shows a summary, with red indicating that a commit introduced an error and green indicating that a commit fixed an error. Use the -e -flag to see the full errors. +flag to see the full errors and -l to see which boards caused which errors. If you really want to see build results as they happen, use -v when doing a -build (and -e if you want to see errors as well). +build (-e will be enabled automatically). You don't need to stick around on that branch while buildman is running. It checks out its own copy of the source code, so you can change branches,
tools/buildman/builder.py +42 -9
116 lines not on this page · raw message
tools/buildman/buildman.py +2 -0
@@ -94,6 +94,8 @@ parser.add_option('-j', '--jobs', dest='jobs', type='int', default=None, help='Number of jobs to run at once (passed to make)') parser.add_option('-k', '--keep-outputs', action='store_true', default=False, help='Keep all build output files (e.g. binaries)') +parser.add_option('-l', '--list-error-boards', action='store_true', + default=False, help='Show a list of boards next to each error/warning') parser.add_option('--list-tool-chains', action='store_true', default=False, help='List available tool chains') parser.add_option('-n', '--dry-run', action='store_true', dest='dry_run',
tools/buildman/control.py +2 -1
@@ -218,7 +218,8 @@ def DoBuildman(options, args): options) builder.SetDisplayOptions(options.show_errors, options.show_sizes, - options.show_detail, options.show_bloat) + options.show_detail, options.show_bloat, + options.list_error_boards) if options.summary: # We can't show function sizes without board details at present if options.show_bloat:
Signature · 1 line
2.1.0.rc2.206.gedb03e5
PATCH v9 11/14 buildman: Add an option to show which boards caused which errors
Quoted 13 lines · Simon Glass
Add a -l option to display a list of offending boards against each error/warning line. The information will be shown in brackets as below:
02: wip
sandbox: + sandbox
arm: + seaboard(quoted diff) +6
-0+(sandbox) arch/sandbox/cpu/cpu.c: In function 'timer_get_us': +(sandbox) arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable] +(seaboard) board/nvidia/seaboard/seaboard.c: In function 'pin_mux_mmc': +(seaboard) board/nvidia/seaboard/seaboard.c:36:9: warning: unused variable 'fred' [-Wunused-variable] +(seaboard) int fred; +(seaboard) ^
Quoted 1 line
Signed-off-by Simon Glass <[email protected]>
Doing buildman ... ; buildman ... -svl (since I want to spit out the summary at once due to how LSF mangles output to file), I don't see anything for what caused powerpc failures say (where there's lot of things complaining about generic board). So something isn't right, is this something you can reproduce locally? I'm going to make sure that -e really is implied like the doc says. Thanks!
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140825/7aafa960/attachment.pgp>
PATCH v9 11/14 buildman: Add an option to show which boards caused which errors
Quoted 14 lines · Tom Rini
Quoted 13 lines · Simon Glass
Add a -l option to display a list of offending boards against each error/warning line. The information will be shown in brackets as below:
02: wip
sandbox: + sandbox
arm: + seaboard(quoted diff) +6
-0+(sandbox) arch/sandbox/cpu/cpu.c: In function 'timer_get_us': +(sandbox) arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable] +(seaboard) board/nvidia/seaboard/seaboard.c: In function 'pin_mux_mmc': +(seaboard) board/nvidia/seaboard/seaboard.c:36:9: warning: unused variable 'fred' [-Wunused-variable] +(seaboard) int fred; +(seaboard) ^
Quoted 8 lines
Quoted 1 line
Signed-off-by Simon Glass <[email protected]>Doing buildman ... ; buildman ... -svl (since I want to spit out the summary at once due to how LSF mangles output to file), I don't see anything for what caused powerpc failures say (where there's lot of things complaining about generic board). So something isn't right, is this something you can reproduce locally? I'm going to make sure that -e really is implied like the doc says. Thanks!
yeah, OK, the doc is wrong and it needs to have -e passed as well.
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140826/b0472ccc/attachment.pgp>
PATCH v9 11/14 buildman: Add an option to show which boards caused which errors
In reply to Tom RiniHi Tom,
Quoted 14 lines · Tom Rini
Quoted 13 lines · Simon Glass
Add a -l option to display a list of offending boards against each error/warning line. The information will be shown in brackets as below:
02: wip
sandbox: + sandbox
arm: + seaboard(quoted diff) +6
-0+(sandbox) arch/sandbox/cpu/cpu.c: In function 'timer_get_us': +(sandbox) arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable] +(seaboard) board/nvidia/seaboard/seaboard.c: In function 'pin_mux_mmc': +(seaboard) board/nvidia/seaboard/seaboard.c:36:9: warning: unused variable 'fred' [-Wunused-variable] +(seaboard) int fred; +(seaboard) ^
Quoted 4 lines
Quoted 1 line
Signed-off-by Simon Glass <[email protected]>Doing buildman ... ; buildman ... -svl (since I want to spit out the summary at once due to how LSF mangles output to file), I don't see
What is -s?
Quoted 4 lines
anything for what caused powerpc failures say (where there's lot of things complaining about generic board). So something isn't right, is this something you can reproduce locally? I'm going to make sure that -e really is implied like the doc says. Thanks!
./tools/buildman/buildman -vl sandbox
Building current source for 1 boards (1 thread, 4 jobs per thread)
sandbox: + sandbox
+(sandbox) arch/sandbox/cpu/cpu.c: In function ?__udelay?:
+(sandbox) arch/sandbox/cpu/cpu.c:36:5: error: unknown type name ?oijew? +(sandbox) arch/sandbox/cpu/cpu.c:37:2: error: conflicting types for ?os_usleep? +(sandbox) include/os.h:155:6: note: previous declaration of ?os_usleep? was here
+(sandbox) make[2]: *** [arch/sandbox/cpu/cpu.o] Error 1
+(sandbox) make[1]: *** [arch/sandbox/cpu] Error 2
+(sandbox) make: *** [sub-make] Error 2
w+(sandbox) arch/sandbox/cpu/cpu.c: In function ?do_reset?:
w+(sandbox) arch/sandbox/cpu/cpu.c:27:13: warning: unused variable ?i? [-Wunused-variable]
w+(sandbox) arch/sandbox/cpu/cpu.c:37:2: warning: parameter names (without types) in function declaration [enabled by default]
0 0 1 /1 sandbox
This shows warnings and errors, and -e is implied (but note you should test with both series - I pushed 'buildman5' to u-boot-x86.git just in case you only have one).
I might be able to repeat your problem, except for the LSF part. What command line are you using?
Regards,
Simon
PATCH v9 11/14 buildman: Add an option to show which boards caused which errors
Quoted 17 lines · Simon Glass
Hi Tom,
Quoted 14 lines · Tom Rini
Quoted 13 lines · Simon Glass
Add a -l option to display a list of offending boards against each error/warning line. The information will be shown in brackets as below:
02: wip
sandbox: + sandbox
arm: + seaboard(quoted diff) +6
-0+(sandbox) arch/sandbox/cpu/cpu.c: In function 'timer_get_us': +(sandbox) arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable] +(seaboard) board/nvidia/seaboard/seaboard.c: In function 'pin_mux_mmc': +(seaboard) board/nvidia/seaboard/seaboard.c:36:9: warning: unused variable 'fred' [-Wunused-variable] +(seaboard) int fred; +(seaboard) ^
Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Signed-off-by Simon Glass <[email protected]>Doing buildman ... ; buildman ... -svl (since I want to spit out the summary at once due to how LSF mangles output to file), I don't see
What is -s?
summary.
Quoted 23 lines
Quoted 4 lines
anything for what caused powerpc failures say (where there's lot of things complaining about generic board). So something isn't right, is this something you can reproduce locally? I'm going to make sure that -e really is implied like the doc says. Thanks!
./tools/buildman/buildman-vl sandbox
Building current source for 1 boards (1 thread, 4 jobs per thread)
sandbox: + sandbox(quoted diff) +4
-0+(sandbox) arch/sandbox/cpu/cpu.c: In function ?__udelay?: +(sandbox) arch/sandbox/cpu/cpu.c:36:5: error: unknown type name ?oijew? +(sandbox) arch/sandbox/cpu/cpu.c:37:2: error: conflicting types for ?os_usleep? +(sandbox) include/os.h:155:6: note: previous declaration of?os_usleep? was here
(quoted diff) +3
-0+(sandbox) make[2]: *** [arch/sandbox/cpu/cpu.o] Error 1 +(sandbox) make[1]: *** [arch/sandbox/cpu] Error 2 +(sandbox) make: *** [sub-make] Error 2w+(sandbox)
arch/sandbox/cpu/cpu.c: In function ?do_reset?:
w+(sandbox)arch/sandbox/cpu/cpu.c:27:13: warning: unused variable ?i? [-Wunused-variable]
w+(sandbox)arch/sandbox/cpu/cpu.c:37:2: warning: parameter names (without types) in function declaration [enabled by default]
0 0 1 /1 sandbox
Quoted 6 lines
This shows warnings and errors, and -e is implied (but note you should test with both series - I pushed 'buildman5' to u-boot-x86.git just in case you only have one).
I might be able to repeat your problem, except for the LSF part. What command line are you using?
I do:
./tools/buildman/buildman -b master -c 1 -T 1 -j 24 \
-o /tmp/trini/eldk521 -G ~/.buildman.eldk521 'arm|powerpc'
./tools/buildman/buildman -b master -c 1 -T 1 -j 24 \
-o /tmp/trini/eldk521 -G ~/.buildman.eldk521 'arm|powerpc' -svelDoing '-svl' would omit the information I expected, adding in -e worked. I do it this way since LSF only lets me log raw stdout to a file so doing a build then a summary pass gives me more easily readable output.
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140828/9df4ad82/attachment.pgp>
PATCH v9 11/14 buildman: Add an option to show which boards caused which errors
Hi Tom,
Quoted 58 lines · Tom Rini
Quoted 17 lines · Simon Glass
Hi Tom,
Quoted 14 lines · Tom Rini
Quoted 13 lines · Simon Glass
Add a -l option to display a list of offending boards against each error/warning line. The information will be shown in brackets as below:
02: wip
sandbox: + sandbox
arm: + seaboard(quoted diff) +6
-0+(sandbox) arch/sandbox/cpu/cpu.c: In function 'timer_get_us': +(sandbox) arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable] +(seaboard) board/nvidia/seaboard/seaboard.c: In function 'pin_mux_mmc': +(seaboard) board/nvidia/seaboard/seaboard.c:36:9: warning: unused variable 'fred' [-Wunused-variable] +(seaboard) int fred; +(seaboard) ^Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Signed-off-by Simon Glass <[email protected]>Doing buildman ... ; buildman ... -svl (since I want to spit out the summary at once due to how LSF mangles output to file), I don't see
What is -s?
summary.
Quoted 23 lines
Quoted 4 lines
anything for what caused powerpc failures say (where there's lot of things complaining about generic board). So something isn't right, is this something you can reproduce locally? I'm going to make sure that -e really is implied like the doc says. Thanks!
./tools/buildman/buildman-vl sandbox
Building current source for 1 boards (1 thread, 4 jobs per thread)
sandbox: + sandbox(quoted diff) +4
-0+(sandbox) arch/sandbox/cpu/cpu.c: In function ?__udelay?: +(sandbox) arch/sandbox/cpu/cpu.c:36:5: error: unknown type name ?oijew? +(sandbox) arch/sandbox/cpu/cpu.c:37:2: error: conflicting types for ?os_usleep? +(sandbox) include/os.h:155:6: note: previous declaration of?os_usleep? was here
(quoted diff) +3
-0+(sandbox) make[2]: *** [arch/sandbox/cpu/cpu.o] Error 1 +(sandbox) make[1]: *** [arch/sandbox/cpu] Error 2 +(sandbox) make: *** [sub-make] Error 2w+(sandbox)
arch/sandbox/cpu/cpu.c: In function ?do_reset?:
w+(sandbox)arch/sandbox/cpu/cpu.c:27:13: warning: unused variable ?i? [-Wunused-variable]
w+(sandbox)arch/sandbox/cpu/cpu.c:37:2: warning: parameter names (without types) in function declaration [enabled by default]
0 0 1 /1 sandboxQuoted 6 lines
This shows warnings and errors, and -e is implied (but note you should test with both series - I pushed 'buildman5' to u-boot-x86.git just in case you only have one).
I might be able to repeat your problem, except for the LSF part. What command line are you using?
I do:
Quoted 8 lines
./tools/buildman/buildman -b master -c 1 -T 1 -j 24 \ -o /tmp/trini/eldk521 -G ~/.buildman.eldk521 'arm|powerpc' ./tools/buildman/buildman -b master -c 1 -T 1 -j 24 \ -o /tmp/trini/eldk521 -G ~/.buildman.eldk521 'arm|powerpc' -svelDoing '-svl' would omit the information I expected, adding in -e worked. I do it this way since LSF only lets me log raw stdout to a file so doing a build then a summary pass gives me more easily readable output.
OK I missed that you were doing two steps.
-v is only meaningful *without* -s: it makes the *build* verbose. When building, -v implies -e,
The summary (-s) is always verbose, so doesn't need -v.
Regards,
Simon
PATCH v9 11/14 buildman: Add an option to show which boards caused which errors
Quoted 70 lines · Simon Glass
Hi Tom,
Quoted 58 lines · Tom Rini
Quoted 17 lines · Simon Glass
Hi Tom,
Quoted 14 lines · Tom Rini
Quoted 13 lines · Simon Glass
Add a -l option to display a list of offending boards against each error/warning line. The information will be shown in brackets as below:
02: wip
sandbox: + sandbox
arm: + seaboard(quoted diff) +6
-0+(sandbox) arch/sandbox/cpu/cpu.c: In function 'timer_get_us': +(sandbox) arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable] +(seaboard) board/nvidia/seaboard/seaboard.c: In function 'pin_mux_mmc': +(seaboard) board/nvidia/seaboard/seaboard.c:36:9: warning: unused variable 'fred' [-Wunused-variable] +(seaboard) int fred; +(seaboard) ^Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Signed-off-by Simon Glass <[email protected]>Doing buildman ... ; buildman ... -svl (since I want to spit out the summary at once due to how LSF mangles output to file), I don't see
What is -s?
summary.
Quoted 23 lines
Quoted 4 lines
anything for what caused powerpc failures say (where there's lot of things complaining about generic board). So something isn't right, is this something you can reproduce locally? I'm going to make sure that -e really is implied like the doc says. Thanks!
./tools/buildman/buildman-vl sandbox
Building current source for 1 boards (1 thread, 4 jobs per thread)
sandbox: + sandbox(quoted diff) +4
-0+(sandbox) arch/sandbox/cpu/cpu.c: In function ?__udelay?: +(sandbox) arch/sandbox/cpu/cpu.c:36:5: error: unknown type name ?oijew? +(sandbox) arch/sandbox/cpu/cpu.c:37:2: error: conflicting types for ?os_usleep? +(sandbox) include/os.h:155:6: note: previous declaration of?os_usleep? was here
(quoted diff) +3
-0+(sandbox) make[2]: *** [arch/sandbox/cpu/cpu.o] Error 1 +(sandbox) make[1]: *** [arch/sandbox/cpu] Error 2 +(sandbox) make: *** [sub-make] Error 2w+(sandbox)
arch/sandbox/cpu/cpu.c: In function ?do_reset?:
w+(sandbox)arch/sandbox/cpu/cpu.c:27:13: warning: unused variable ?i? [-Wunused-variable]
w+(sandbox)arch/sandbox/cpu/cpu.c:37:2: warning: parameter names (without types) in function declaration [enabled by default]
0 0 1 /1 sandboxQuoted 6 lines
This shows warnings and errors, and -e is implied (but note you should test with both series - I pushed 'buildman5' to u-boot-x86.git just in case you only have one).
I might be able to repeat your problem, except for the LSF part. What command line are you using?
I do:
Quoted 8 lines
./tools/buildman/buildman -b master -c 1 -T 1 -j 24 \ -o /tmp/trini/eldk521 -G ~/.buildman.eldk521 'arm|powerpc' ./tools/buildman/buildman -b master -c 1 -T 1 -j 24 \ -o /tmp/trini/eldk521 -G ~/.buildman.eldk521 'arm|powerpc' -svelDoing '-svl' would omit the information I expected, adding in -e worked. I do it this way since LSF only lets me log raw stdout to a file so doing a build then a summary pass gives me more easily readable output.
Quoted 6 lines
OK I missed that you were doing two steps.
-v is only meaningful *without* -s: it makes the *build* verbose. When building, -v implies -e,
The summary (-s) is always verbose, so doesn't need -v.
OK, but in step #2, -svl _doesn't_ give me information about what boards have what problem and -svel does. You're saying -sel should work too, but commit text (and help text?) says that -l implies -e, but it's not.
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140901/ed77660d/attachment.pgp>
PATCH v9 11/14 buildman: Add an option to show which boards caused which errors
Hi Tom,
Quoted 73 lines · Tom Rini
Quoted 72 lines · Simon Glass
Hi Tom,
Quoted 58 lines · Tom Rini
Quoted 17 lines · Simon Glass
Hi Tom,
Quoted 14 lines · Tom Rini
Quoted 13 lines · Simon Glass
Add a -l option to display a list of offending boards against each error/warning line. The information will be shown in brackets as below:
02: wip
sandbox: + sandbox
arm: + seaboard(quoted diff) +6
-0+(sandbox) arch/sandbox/cpu/cpu.c: In function 'timer_get_us': +(sandbox) arch/sandbox/cpu/cpu.c:40:9: warning: unused variable 'i' [-Wunused-variable] +(seaboard) board/nvidia/seaboard/seaboard.c: In function 'pin_mux_mmc': +(seaboard) board/nvidia/seaboard/seaboard.c:36:9: warning: unused variable 'fred' [-Wunused-variable] +(seaboard) int fred; +(seaboard) ^Quoted 6 lines
Quoted 4 lines
Quoted 1 line
Signed-off-by Simon Glass <[email protected]>Doing buildman ... ; buildman ... -svl (since I want to spit out the summary at once due to how LSF mangles output to file), I don't see
What is -s?
summary.
Quoted 23 lines
Quoted 4 lines
anything for what caused powerpc failures say (where there's lot of things complaining about generic board). So something isn't right, is this something you can reproduce locally? I'm going to make sure that -e really is implied like the doc says. Thanks!
./tools/buildman/buildman-vl sandbox
Building current source for 1 boards (1 thread, 4 jobs per thread)
sandbox: + sandbox(quoted diff) +4
-0+(sandbox) arch/sandbox/cpu/cpu.c: In function ?__udelay?: +(sandbox) arch/sandbox/cpu/cpu.c:36:5: error: unknown type name ?oijew? +(sandbox) arch/sandbox/cpu/cpu.c:37:2: error: conflicting types for ?os_usleep? +(sandbox) include/os.h:155:6: note: previous declaration of?os_usleep? was here
(quoted diff) +3
-0+(sandbox) make[2]: *** [arch/sandbox/cpu/cpu.o] Error 1 +(sandbox) make[1]: *** [arch/sandbox/cpu] Error 2 +(sandbox) make: *** [sub-make] Error 2w+(sandbox)
arch/sandbox/cpu/cpu.c: In function ?do_reset?:
w+(sandbox)arch/sandbox/cpu/cpu.c:27:13: warning: unused variable ?i? [-Wunused-variable]
w+(sandbox)arch/sandbox/cpu/cpu.c:37:2: warning: parameter names (without types) in function declaration [enabled by default]
0 0 1 /1 sandboxQuoted 6 lines
This shows warnings and errors, and -e is implied (but note you should test with both series - I pushed 'buildman5' to u-boot-x86.git just in case you only have one).
I might be able to repeat your problem, except for the LSF part. What command line are you using?
I do:
Quoted 8 lines
./tools/buildman/buildman -b master -c 1 -T 1 -j 24 \ -o /tmp/trini/eldk521 -G ~/.buildman.eldk521 'arm|powerpc' ./tools/buildman/buildman -b master -c 1 -T 1 -j 24 \ -o /tmp/trini/eldk521 -G ~/.buildman.eldk521 'arm|powerpc' -svelDoing '-svl' would omit the information I expected, adding in -e worked. I do it this way since LSF only lets me log raw stdout to a file so doing a build then a summary pass gives me more easily readable output.
OK I missed that you were doing two steps.
Quoted 8 lines
Quoted 4 lines
-v is only meaningful *without* -s: it makes the *build* verbose. When building, -v implies -e,
The summary (-s) is always verbose, so doesn't need -v.
OK, but in step #2, -svl _doesn't_ give me information about what boards have what problem and -svel does. You're saying -sel should work too, but commit text (and help text?) says that -l implies -e, but it's not.
-v implies -e if you are building (no -s)
-v means nothing if you are not building (-s)
I don't think -l implies anything. I suspect I have stuffed up something in the docs but I can't see what, so will await your response. Sorry for the trouble.
Regards,
Simon
PATCH v9 12/14 buildman: Separate out display of warnings and errors
In reply to Simon GlassSome boards unfortunately build with warnings and it is useful to be able to easily distinguish the warnings from the errors.
Use a simple pattern match to categorise gcc output into warnings and errors, and display each separately. New warnings are shown in magenta (with a w+ prefix) and fixed warnings are shown in yellow with a w- prefix.
---
Changes in v9:
- Add new patch to separate out display of warnings and errors
Changes in v8: None
Changes in v7: None
Changes in v6: None
Changes in v5: None
tools/buildman/README | 3 ++ tools/buildman/builder.py | 105 ++++++++++++++++++++++++++++++++++++---------- 2 files changed, 85 insertions(+), 23 deletions(-)
tools/buildman/README +3 -0
@@ -445,6 +445,9 @@ If many boards have the same error, then -e will display the error only once. This makes the output as concise as possible. To see which boards have each error, use -l. +Buildman tries to distinguish warnings from errors, and shows warning lines +separately with a 'w' prefix. + The full build output in this case is available in: ../lcd9b/12_of_18_gd92aff7_lcd--Add-support-for/lubbock/
tools/buildman/builder.py +82 -23
201 lines not on this page · raw message
Signature · 1 line
2.1.0.rc2.206.gedb03e5
PATCH v9 13/14 patman: Fix detection of git version
In reply to Simon GlassA missing 'global' declaration means that this feature does not currently work. Fix it.
---
Changes in v9:
- Add new patch to fix detection of git version
Changes in v8: None
Changes in v7: None
Changes in v6: None
Changes in v5: None
tools/patman/gitutil.py | 2 ++ 1 file changed, 2 insertions(+)
tools/patman/gitutil.py +2 -0
@@ -481,6 +481,8 @@ def GetDefaultUserEmail(): def Setup(): """Set up git utils, by reading the alias files.""" # Check for a git alias file also + global use_no_decorate + alias_fname = GetAliasFile() if alias_fname: settings.ReadGitAliases(alias_fname)
Signature · 1 line
2.1.0.rc2.206.gedb03e5
PATCH v9 14/14 RFC: Deprecate MAKEALL
In reply to Simon GlassSince buildman now includes most of the features of MAKEALL it is probably time to talk about deprecating MAKEALL.
Comments welcome.
---
Changes in v9: None
Changes in v8: None
Changes in v7:
- Remove already-applied patches from the series
- Add the deprecation message at the end of the build also
- Drop the 'colour' patch sadly
Changes in v6: None
Changes in v5:
- Drop patch to search for *cc instead of *gcc for the compiler
MAKEALL | 10 ++++++++++ 1 file changed, 10 insertions(+)
MAKEALL +10 -0
@@ -60,6 +60,14 @@ usage() exit ${ret} } +deprecation() { + echo "** Note: MAKEALL is deprecated - please use buildman instead" + echo "** See tools/buildman/README for details" + echo +} + +deprecation + SHORT_OPTS="ha:c:v:s:b:lmMCnr" LONG_OPTS="help,arch:,cpu:,vendor:,soc:,board:,list,maintainers,mails,check,continue,rebuild-errors"
@@ -849,6 +857,8 @@ print_stats() { kill_children fi + deprecation + exit $RC }
Signature · 1 line
2.1.0.rc2.206.gedb03e5
v9 cover Add some missing buildman features and deprecate MAKEALL
In reply to Simon GlassQuoted 6 lines · Simon Glass
Buildman has been around for a little over a year and is used by a fair number of U-Boot developers. However quite a few people still use MAKEALL.
Buildman was intended to replace MAKEALL, so perhaps now is a good time to start that process.
Quoted 8 lines
The reasons to deprecate MAKEALL are:
- We don't want to maintain two build systems
- Buildman is typically faster
- Buildman has a lot more featuresThis series adds a few features to buildman to fill some gaps, adds some information into the README on how to migrate from MAKEALL, and adds a deprecation message to MAKEALL.
I'm testing this out again in the Ubuntu 10.04 / LSF environment I have access to but so far I'm still finding that test for --no-decorate isn't working and I have to manually set it to False.
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140825/c368ef17/attachment.pgp>
v9 cover Add some missing buildman features and deprecate MAKEALL
Hi Tom,
Quoted 13 lines · Tom Rini
Quoted 11 lines · Simon Glass
Buildman has been around for a little over a year and is used by a fair number of U-Boot developers. However quite a few people still use MAKEALL.
Buildman was intended to replace MAKEALL, so perhaps now is a good time to start that process.
The reasons to deprecate MAKEALL are:
- We don't want to maintain two build systems
- Buildman is typically faster
- Buildman has a lot more features
Quoted 7 lines
Quoted 3 lines
This series adds a few features to buildman to fill some gaps, adds some information into the README on how to migrate from MAKEALL, and adds a deprecation message to MAKEALL.
I'm testing this out again in the Ubuntu 10.04 / LSF environment I have access to but so far I'm still finding that test for --no-decorate isn't working and I have to manually set it to False.
Oh dear. I wrote a test for this in the second series. So if you apply that series, do the tests pass? It is in func_test.py and is called testGitSetup().
Regards,
Simon
v9 cover Add some missing buildman features and deprecate MAKEALL
Quoted 20 lines · Simon Glass
Hi Tom,
Quoted 17 lines · Tom Rini
Quoted 14 lines · Simon Glass
Buildman has been around for a little over a year and is used by a fair number of U-Boot developers. However quite a few people still use MAKEALL.
Buildman was intended to replace MAKEALL, so perhaps now is a good time to start that process.
The reasons to deprecate MAKEALL are:
- We don't want to maintain two build systems
- Buildman is typically faster
- Buildman has a lot more featuresThis series adds a few features to buildman to fill some gaps, adds some information into the README on how to migrate from MAKEALL, and adds a deprecation message to MAKEALL.
Quoted 7 lines
Quoted 3 lines
I'm testing this out again in the Ubuntu 10.04 / LSF environment I have access to but so far I'm still finding that test for --no-decorate isn't working and I have to manually set it to False.
Oh dear. I wrote a test for this in the second series. So if you apply that series, do the tests pass? It is in
func_test.pyand is calledtestGitSetup().
With my local kludge removed yes, all tests pass.
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140825/36677754/attachment.pgp>
v9 cover Add some missing buildman features and deprecate MAKEALL
Hi Tom,
Quoted 25 lines · Tom Rini
Quoted 24 lines · Simon Glass
Hi Tom,
Quoted 20 lines · Tom Rini
Quoted 6 lines · Simon Glass
Buildman has been around for a little over a year and is used by a fair number of U-Boot developers. However quite a few people still use MAKEALL.
Buildman was intended to replace MAKEALL, so perhaps now is a good time to start that process.
Quoted 8 lines
The reasons to deprecate MAKEALL are:
- We don't want to maintain two build systems
- Buildman is typically faster
- Buildman has a lot more featuresThis series adds a few features to buildman to fill some gaps, adds some information into the README on how to migrate from MAKEALL, and adds a deprecation message to MAKEALL.
I'm testing this out again in the Ubuntu 10.04 / LSF environment I have access to but so far I'm still finding that test for --no-decorate isn't working and I have to manually set it to False.
Quoted 5 lines
Quoted 3 lines
Oh dear. I wrote a test for this in the second series. So if you apply that series, do the tests pass? It is in
func_test.pyand is calledtestGitSetup().With my local kludge removed yes, all tests pass.
It's quite mysterious. I installed Ubuntu 10.04 and it runs python 2.6.5.
The --no-decorate patch does work, although it does print an error. I could suppress that easily enough, just adding another arg 'capture_stderr=True' to the RunPipe() call in gitutil.py.Setup().
I see a problem with timedelta not having total_seconds. After that the threading dies. So I'm quite impressed you can get buildman to run at all. Are you using one of the later 10.04 releases?
Regards,
Simon
v9 cover Add some missing buildman features and deprecate MAKEALL
Quoted 36 lines · Simon Glass
Hi Tom,
Quoted 25 lines · Tom Rini
Quoted 24 lines · Simon Glass
Hi Tom,
Quoted 20 lines · Tom Rini
Quoted 6 lines · Simon Glass
Buildman has been around for a little over a year and is used by a fair number of U-Boot developers. However quite a few people still use MAKEALL.
Buildman was intended to replace MAKEALL, so perhaps now is a good time to start that process.
Quoted 8 lines
The reasons to deprecate MAKEALL are:
- We don't want to maintain two build systems
- Buildman is typically faster
- Buildman has a lot more featuresThis series adds a few features to buildman to fill some gaps, adds some information into the README on how to migrate from MAKEALL, and adds a deprecation message to MAKEALL.
I'm testing this out again in the Ubuntu 10.04 / LSF environment I have access to but so far I'm still finding that test for --no-decorate isn't working and I have to manually set it to False.
Quoted 5 lines
Quoted 3 lines
Oh dear. I wrote a test for this in the second series. So if you apply that series, do the tests pass? It is in
func_test.pyand is calledtestGitSetup().With my local kludge removed yes, all tests pass.
It's quite mysterious. I installed Ubuntu 10.04 and it runs python 2.6.5.
Quoted 7 lines
The --no-decorate patch does work, although it does print an error. I could suppress that easily enough, just adding another arg 'capture_stderr=True' to the
RunPipe()call ingitutil.py.Setup().I see a problem with timedelta not having
total_seconds. After that the threading dies. So I'm quite impressed you can get buildman to run at all. Are you using one of the later 10.04 releases?
*shakes fist at corporate IT*
The box I use to launch jobs is Ubuntu 10.04.4. The boxes the jobs run on are actually Ubuntu 12.04.1 (I had assumed it was all the same for some reason.
They also have git 1.7.9.5.
Signature · 8 lines
Tom -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 836 bytes Desc: Digital signature URL: <http://lists.denx.de/pipermail/u-boot/attachments/20140828/bc6274d9/attachment.pgp>
v9 cover Add some missing buildman features and deprecate MAKEALL
Hi Tom,
Quoted 45 lines · Tom Rini
Quoted 36 lines · Simon Glass
Hi Tom,
Quoted 25 lines · Tom Rini
Quoted 24 lines · Simon Glass
Hi Tom,
Quoted 20 lines · Tom Rini
Quoted 6 lines · Simon Glass
Buildman has been around for a little over a year and is used by a fair number of U-Boot developers. However quite a few people still use MAKEALL.
Buildman was intended to replace MAKEALL, so perhaps now is a good time to start that process.
Quoted 8 lines
The reasons to deprecate MAKEALL are:
- We don't want to maintain two build systems
- Buildman is typically faster
- Buildman has a lot more featuresThis series adds a few features to buildman to fill some gaps, adds some information into the README on how to migrate from MAKEALL, and adds a deprecation message to MAKEALL.
I'm testing this out again in the Ubuntu 10.04 / LSF environment I have access to but so far I'm still finding that test for --no-decorate isn't working and I have to manually set it to False.
Quoted 5 lines
Quoted 3 lines
Oh dear. I wrote a test for this in the second series. So if you apply that series, do the tests pass? It is in
func_test.pyand is calledtestGitSetup().With my local kludge removed yes, all tests pass.
It's quite mysterious. I installed Ubuntu 10.04 and it runs python 2.6.5.
Quoted 7 lines
The --no-decorate patch does work, although it does print an error. I could suppress that easily enough, just adding another arg 'capture_stderr=True' to the
RunPipe()call ingitutil.py.Setup().I see a problem with timedelta not having
total_seconds. After that the threading dies. So I'm quite impressed you can get buildman to run at all. Are you using one of the later 10.04 releases?
Quoted 7 lines
*shakes fist at corporate IT*
The box I use to launch jobs is Ubuntu 10.04.4. The boxes the jobs run on are actually Ubuntu 12.04.1 (I had assumed it was all the same for some reason.
They also have git 1.7.9.5.
Well they obviously can't make it *too* easy. So where does this leave us in trying to figure this out?
Regards,
Simon
v9 cover Add some missing buildman features and deprecate MAKEALL
Hi Tom,
Quoted 59 lines · Simon Glass
Hi Tom,
Quoted 11 lines · Tom Rini
Quoted 10 lines · Simon Glass
Hi Tom,
Quoted 7 lines · Tom Rini
Quoted 6 lines · Simon Glass
Hi Tom,
Quoted 3 lines · Tom Rini
Quoted 1 line · Simon Glass
Buildman has been around for a little over a year and is used by
a fair
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
number of U-Boot developers. However quite a few people still use
MAKEALL.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Buildman was intended to replace MAKEALL, so perhaps now is a
good time to
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
start that process.
The reasons to deprecate MAKEALL are:
- We don't want to maintain two build systems
- Buildman is typically faster
- Buildman has a lot more featuresThis series adds a few features to buildman to fill some gaps,
adds some
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
information into the README on how to migrate from MAKEALL, and
adds a
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
deprecation message to MAKEALL.
I'm testing this out again in the Ubuntu 10.04 / LSF environment I
have
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
access to but so far I'm still finding that test for --no-decorate
isn't
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
working and I have to manually set it to False.
Oh dear. I wrote a test for this in the second series. So if you
apply
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
that series, do the tests pass? It is in
func_test.pyand is calledtestGitSetup().With my local kludge removed yes, all tests pass.
It's quite mysterious. I installed Ubuntu 10.04 and it runs python
2.6.5.
Quoted 11 lines
Quoted 8 lines
The --no-decorate patch does work, although it does print an error. I could suppress that easily enough, just adding another arg 'capture_stderr=True' to the
RunPipe()call ingitutil.py.Setup().I see a problem with timedelta not having
total_seconds. After that the threading dies. So I'm quite impressed you can get buildman to run at all. Are you using one of the later 10.04 releases?*shakes fist at corporate IT*
Quoted 9 lines
Quoted 5 lines
The box I use to launch jobs is Ubuntu 10.04.4. The boxes the jobs run on are actually Ubuntu 12.04.1 (I had assumed it was all the same for some reason.
They also have git 1.7.9.5.
Well they obviously can't make it *too* easy. So where does this leave us in trying to figure this out?
It feels like we should probably get the current patches in and see where this leaves us. I'll work on a pull request, unless you think we should wait.
Regards,
Simon