Recursive show-depends and show-rdepends
v2 cover Recursive show-depends and show-rdepends
Hello,
This is a new iteration of the patch from George Redivo [1] that added the <pkg>-show-rrdepends target. This series pushes this much further:
- It starts by doing a cleanup pass on graph-depends, moving all global code into proper functions, and removing most global variables.
- It adds a --quiet option to ask graph-depends to not display its debugging messages, which are not really meaningful for show-recursive-depends/show-recursive-rdepend.
- We have chosen to name the target <pkg>-show-recursive-rdepends instead of <pkg>-show-rrdepends, as it is more explicitly, and allows us to also introduce <pkg>-show-recursive-depends.
- The change in graph-depends (introducing the --flat-list feature) and in the package infrastructure are in separate patches.
- The output of show-recursive-(r)depends is now on a single line (instead of one package by line), to match the output of the existing show-depends and show-rdepends targets.
[1] http://patchwork.ozlabs.org/patch/756075/
Thanks!
Thomas
George Redivo (2):
support/scripts/graph-depends: add --flat-list option package/pkg-generic: add <pkg>-show-recursive-(r)depends targets
Thomas Petazzoni (3):
support/scripts/graph-depends: remove global code and most global
variables
support/scripts/graph-depends: use colors instead of colours support/scripts/graph-depends: add --quiet option
Makefile | 4 + docs/manual/common-usage.txt | 4 +- package/pkg-generic.mk | 8 ++ support/scripts/brpkgutil.py | 18 +-- support/scripts/graph-depends | 313 ++++++++++++++++++++++-------------------- 5 files changed, 188 insertions(+), 159 deletions(-)
Signature · 1 line
2.14.3
PATCH v2 1/5 support/scripts/graph-depends: remove global code and most global variables
The graph-depends script had no main() function, and the main code was actually spread between the function definitions, which was a real mess.
This commit moves the global code into a main() function, which allows to more easily follow the flow of the script. The argument parsing code is moved into a parse_args() function.
Most of the global variables are removed, and are instead passed as argument when appropriate. This has the side-effect that the print_pkg_deps() function takes a lot of argument, but this is considered better than tons of global variables.
The global variables that are removed are: max_depth, transitive, mode, root_colour, target_colour, host_colour, outfile, dict_deps, dict_version, stop_list, exclude_list, arrow_dir.
The root_colour/target_colour/host_colour variables are entirely removed, and instead a single colours array is passed, and it's the function using the colors that actually uses the different entries in the array.
The way the print_attrs() function determines if we're display the root node is not is changed. Instead of relying on the package name and the mode (which requires passing the root package name, and the mode), it relies on the depth: when the depth is 0, we're at the root node.
--- support/scripts/graph-depends | 289 +++++++++++++++++++++--------------------- 1 file changed, 144 insertions(+), 145 deletions(-)
support/scripts/graph-depends +144 -145
@@ -31,96 +31,6 @@ import brpkgutil # Modes of operation: MODE_FULL = 1 # draw full dependency graph for all selected packages MODE_PKG = 2 # draw dependency graph for a given package -mode = 0 - -# Limit drawing the dependency graph to this depth. 0 means 'no limit'. -max_depth = 0 - -# Whether to draw the transitive dependencies -transitive = True - -parser = argparse.ArgumentParser(description="Graph packages dependencies") -parser.add_argument("--check-only", "-C", dest="check_only", action="store_true", default=False, - help="Only do the dependency checks (circular deps...)") -parser.add_argument("--outfile", "-o", metavar="OUT_FILE", dest="outfile", - help="File in which to generate the dot representation") -parser.add_argument("--package", '-p', metavar="PACKAGE", - help="Graph the dependencies of PACKAGE") -parser.add_argument("--depth", '-d', metavar="DEPTH", dest="depth", type=int, default=0, - help="Limit the dependency graph to DEPTH levels; 0 means no limit.") -parser.add_argument("--stop-on", "-s", metavar="PACKAGE", dest="stop_list", action="append", - help="Do not graph past this package (can be given multiple times)." + - " Can be a package name or a glob, " + - " 'virtual' to stop on virtual packages, or " + - "'host' to stop on host packages.") -parser.add_argument("--exclude", "-x", metavar="PACKAGE", dest="exclude_list", action="append", - help="Like --stop-on, but do not add PACKAGE to the graph.") -parser.add_argument("--colours", "-c", metavar="COLOR_LIST", dest="colours", - default="lightblue,grey,gainsboro", - help="Comma-separated list of the three colours to use" + - " to draw the top-level package, the target" + - " packages, and the host packages, in this order." + - " Defaults to: 'lightblue,grey,gainsboro'") -parser.add_argument("--transitive", dest="transitive", action='store_true', - default=False) -parser.add_argument("--no-transitive", dest="transitive", action='store_false', - help="Draw (do not draw) transitive dependencies") -parser.add_argument("--direct", dest="direct", action='store_true', default=True, - help="Draw direct dependencies (the default)") -parser.add_argument("--reverse", dest="direct", action='store_false', - help="Draw reverse dependencies") -args = parser.parse_args() - -check_only = args.check_only - -if args.outfile is None: - outfile = sys.stdout -else: - if check_only: - sys.stderr.write("don't specify outfile and check-only at the same time\n") - sys.exit(1) - outfile = open(args.outfile, "w") - -if args.package is None: - mode = MODE_FULL -else: - mode = MODE_PKG - rootpkg = args.package - -max_depth = args.depth - -if args.stop_list is None: - stop_list = [] -else: - stop_list = args.stop_list - -if args.exclude_list is None: - exclude_list = [] -else: - exclude_list = args.exclude_list - -transitive = args.transitive - -if args.direct: - get_depends_func = brpkgutil.get_depends - arrow_dir = "forward" -else: - if mode == MODE_FULL: - sys.stderr.write("--reverse needs a package\n") - sys.exit(1) - get_depends_func = brpkgutil.get_rdepends - arrow_dir = "back" - -# Get the colours: we need exactly three colours, -# so no need not split more than 4 -# We'll let 'dot' validate the colours... -colours = args.colours.split(',', 4) -if len(colours) != 3: - sys.stderr.write("Error: incorrect colour list '%s'\n" % args.colours) - sys.exit(1) -root_colour = colours[0] -target_colour = colours[1] -host_colour = colours[2] allpkgs = []
@@ -145,7 +55,7 @@ def get_targets(): # 'dependencies', which contains tuples of the form (pkg1 -> # pkg2_on_which_pkg1_depends, pkg3 -> pkg4_on_which_pkg3_depends) and # the function finally returns this list. -def get_all_depends(pkgs): +def get_all_depends(pkgs, get_depends_func): dependencies = [] # Filter the packages for which we already have the dependencies
@@ -175,7 +85,7 @@ def get_all_depends(pkgs): deps.add(dep) if len(deps) != 0: - newdeps = get_all_depends(deps) + newdeps = get_all_depends(deps, get_depends_func) if newdeps is not None: dependencies += newdeps
@@ -193,35 +103,6 @@ TARGET_EXCEPTIONS = [ "target-post-image", ] -# In full mode, start with the result of get_targets() to get the main -# targets and then use get_all_depends() for all targets -if mode == MODE_FULL: - targets = get_targets() - dependencies = [] - allpkgs.append('all') - filtered_targets = [] - for tg in targets: - # Skip uninteresting targets - if tg in TARGET_EXCEPTIONS: - continue - dependencies.append(('all', tg)) - filtered_targets.append(tg) - deps = get_all_depends(filtered_targets) - if deps is not None: - dependencies += deps - rootpkg = 'all' - -# In pkg mode, start directly with get_all_depends() on the requested -# package -elif mode == MODE_PKG: - dependencies = get_all_depends([rootpkg]) - -# Make the dependencies a dictionnary { 'pkg':[dep1, dep2, ...] } -dict_deps = {} -for dep in dependencies: - if dep[0] not in dict_deps: - dict_deps[dep[0]] = [] - dict_deps[dep[0]].append(dep[1]) # Basic cache for the results of the is_dep() function, in order to # optimize the execution time. The cache is a dict of dict of boolean
@@ -328,7 +209,7 @@ def check_circular_deps(deps): # This functions trims down the dependency list of all packages. # It applies in sequence all the dependency-elimination methods. -def remove_extra_deps(deps): +def remove_extra_deps(deps, transitive): for pkg in list(deps.keys()): if not pkg == 'all': deps[pkg] = remove_mandatory_deps(pkg, deps)
@@ -338,32 +219,22 @@ def remove_extra_deps(deps): return deps -check_circular_deps(dict_deps) -if check_only: - sys.exit(0) - -dict_deps = remove_extra_deps(dict_deps) -dict_version = brpkgutil.get_version([pkg for pkg in allpkgs - if pkg != "all" and not pkg.startswith("root")]) - - # Print the attributes of a node: label and fill-color -def print_attrs(pkg): +def print_attrs(outfile, pkg, version, depth, colors): name = pkg_node_name(pkg) if pkg == 'all': label = 'ALL' else: label = pkg - if pkg == 'all' or (mode == MODE_PKG and pkg == rootpkg): - color = root_colour + if depth == 0: + color = colors[0] else: if pkg.startswith('host') \ or pkg.startswith('toolchain') \ or pkg.startswith('rootfs'): - color = host_colour + color = colors[2] else: - color = target_colour - version = dict_version.get(pkg) + color = colors[1] if version == "virtual": outfile.write("%s [label = <<I>%s</I>>]\n" % (name, label)) else:
@@ -371,12 +242,16 @@ def print_attrs(pkg): outfile.write("%s [color=%s,style=filled]\n" % (name, color)) +done_deps = [] + + # Print the dependency graph of a package -def print_pkg_deps(depth, pkg): +def print_pkg_deps(outfile, dict_deps, dict_version, stop_list, exclude_list, + arrow_dir, depth, max_depth, pkg, colors): if pkg in done_deps: return done_deps.append(pkg) - print_attrs(pkg) + print_attrs(outfile, pkg, dict_version.get(pkg), depth, colors) if pkg not in dict_deps: return for p in stop_list:
@@ -401,13 +276,137 @@ def print_pkg_deps(depth, pkg): break if add: outfile.write("%s -> %s [dir=%s]\n" % (pkg_node_name(pkg), pkg_node_name(d), arrow_dir)) - print_pkg_deps(depth + 1, d) + print_pkg_deps(outfile, dict_deps, dict_version, stop_list, exclude_list, + arrow_dir, depth + 1, max_depth, d, colors) + + +def parse_args(): + parser = argparse.ArgumentParser(description="Graph packages dependencies") + parser.add_argument("--check-only", "-C", dest="check_only", action="store_true", default=False, + help="Only do the dependency checks (circular deps...)") + parser.add_argument("--outfile", "-o", metavar="OUT_FILE", dest="outfile", + help="File in which to generate the dot representation") + parser.add_argument("--package", '-p', metavar="PACKAGE", + help="Graph the dependencies of PACKAGE") + parser.add_argument("--depth", '-d', metavar="DEPTH", dest="depth", type=int, default=0, + help="Limit the dependency graph to DEPTH levels; 0 means no limit.") + parser.add_argument("--stop-on", "-s", metavar="PACKAGE", dest="stop_list", action="append", + help="Do not graph past this package (can be given multiple times)." + + " Can be a package name or a glob, " + + " 'virtual' to stop on virtual packages, or " + + "'host' to stop on host packages.") + parser.add_argument("--exclude", "-x", metavar="PACKAGE", dest="exclude_list", action="append", + help="Like --stop-on, but do not add PACKAGE to the graph.") + parser.add_argument("--colours", "-c", metavar="COLOR_LIST", dest="colours", + default="lightblue,grey,gainsboro", + help="Comma-separated list of the three colours to use" + + " to draw the top-level package, the target" + + " packages, and the host packages, in this order." + + " Defaults to: 'lightblue,grey,gainsboro'") + parser.add_argument("--transitive", dest="transitive", action='store_true', + default=False) + parser.add_argument("--no-transitive", dest="transitive", action='store_false', + help="Draw (do not draw) transitive dependencies") + parser.add_argument("--direct", dest="direct", action='store_true', default=True, + help="Draw direct dependencies (the default)") + parser.add_argument("--reverse", dest="direct", action='store_false', + help="Draw reverse dependencies") + return parser.parse_args() + + +def main(): + args = parse_args() + + check_only = args.check_only + + if args.outfile is None: + outfile = sys.stdout + else: + if check_only: + sys.stderr.write("don't specify outfile and check-only at the same time\n") + sys.exit(1) + outfile = open(args.outfile, "w") + if args.package is None: + mode = MODE_FULL + else: + mode = MODE_PKG + rootpkg = args.package -# Start printing the graph data -outfile.write("digraph G {\n") + if args.stop_list is None: + stop_list = [] + else: + stop_list = args.stop_list + + if args.exclude_list is None: + exclude_list = [] + else: + exclude_list = args.exclude_list + + if args.direct: + get_depends_func = brpkgutil.get_depends + arrow_dir = "forward" + else: + if mode == MODE_FULL: + sys.stderr.write("--reverse needs a package\n") + sys.exit(1) + get_depends_func = brpkgutil.get_rdepends + arrow_dir = "back" + + # Get the colours: we need exactly three colours, + # so no need not split more than 4 + # We'll let 'dot' validate the colours... + colours = args.colours.split(',', 4) + if len(colours) != 3: + sys.stderr.write("Error: incorrect colour list '%s'\n" % args.colours) + sys.exit(1) + + # In full mode, start with the result of get_targets() to get the main + # targets and then use get_all_depends() for all targets + if mode == MODE_FULL: + targets = get_targets() + dependencies = [] + allpkgs.append('all') + filtered_targets = [] + for tg in targets: + # Skip uninteresting targets + if tg in TARGET_EXCEPTIONS: + continue + dependencies.append(('all', tg)) + filtered_targets.append(tg) + deps = get_all_depends(filtered_targets, get_depends_func) + if deps is not None: + dependencies += deps + rootpkg = 'all' + + # In pkg mode, start directly with get_all_depends() on the requested + # package + elif mode == MODE_PKG: + dependencies = get_all_depends([rootpkg], get_depends_func) + + # Make the dependencies a dictionnary { 'pkg':[dep1, dep2, ...] } + dict_deps = {} + for dep in dependencies: + if dep[0] not in dict_deps: + dict_deps[dep[0]] = [] + dict_deps[dep[0]].append(dep[1]) + + check_circular_deps(dict_deps) + if check_only: + sys.exit(0) + + dict_deps = remove_extra_deps(dict_deps, args.transitive) + dict_version = brpkgutil.get_version([pkg for pkg in allpkgs + if pkg != "all" and not pkg.startswith("root")]) + + # Start printing the graph data + outfile.write("digraph G {\n") + + print_pkg_deps(outfile, dict_deps, dict_version, stop_list, exclude_list, + arrow_dir, 0, args.depth, rootpkg, colours) + + outfile.write("}\n") -done_deps = [] -print_pkg_deps(0, rootpkg) -outfile.write("}\n") +if __name__ == "__main__": + sys.exit(main())
Signature · 1 line
2.14.3
PATCH v2 1/5 support/scripts/graph-depends: remove global code and most global variables
Quoted 24 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Thomas" == Thomas Petazzoni <[email protected]> writes:
The graph-depends script had no
main()function, and the main code was actually spread between the function definitions, which was a real mess.This commit moves the global code into a
main()function, which allows to more easily follow the flow of the script. The argument parsing code is moved into aparse_args()function.Most of the global variables are removed, and are instead passed as argument when appropriate. This has the side-effect that the
print_pkg_deps()function takes a lot of argument, but this is considered better than tons of global variables.The global variables that are removed are:
max_depth, transitive, mode,root_colour,target_colour,host_colour, outfile,dict_deps,dict_version,stop_list,exclude_list,arrow_dir.The root_colour/target_colour/host_colour variables are entirely removed, and instead a single colours array is passed, and it's the function using the colors that actually uses the different entries in the array.
Quoted 7 lines
The way the
print_attrs()function determines if we're display the root node is not is changed. Instead of relying on the package name and the mode (which requires passing the root package name, and the mode), it relies on the depth: when the depth is 0, we're at the root node.Signed-off-by Thomas Petazzoni <[email protected]>
Committed, thanks.
Signature · 1 line
Bye, Peter Korsgaard
PATCH v2 2/5 support/scripts/graph-depends: use colors instead of colours
In reply to Thomas PetazzoniThe graph-depends was not very consistent in colors vs. colours: some parts were using colours, some parts were using colors.
Let's settle on the US spelling, colors.
This change the user-visble option --colours to --colors, but it is unlikely that a lot of users customize the colors through BR2_GRAPH_DEPS_OPTS, so this user interface change is considered reasonable.
--- docs/manual/common-usage.txt | 4 ++-- support/scripts/graph-depends | 16 ++++++++-------- 2 files changed, 10 insertions(+), 10 deletions(-)
docs/manual/common-usage.txt +2 -2
@@ -224,12 +224,12 @@ The +graph-depends+ behaviour can be controlled by setting options in the * +--transitive+, +--no-transitive+, to draw (or not) the transitive dependencies. The default is to not draw transitive dependencies. -* +--colours R,T,H+, the comma-separated list of colours to draw the +* +--colors R,T,H+, the comma-separated list of colors to draw the root package (+R+), the target packages (+T+) and the host packages (+H+). Defaults to: +lightblue,grey,gainsboro+ -------------------------------- -BR2_GRAPH_DEPS_OPTS='-d 3 --no-transitive --colours=red,green,blue' make graph-depends +BR2_GRAPH_DEPS_OPTS='-d 3 --no-transitive --colors=red,green,blue' make graph-depends -------------------------------- === Graphing the build duration
support/scripts/graph-depends +8 -8
@@ -297,9 +297,9 @@ def parse_args(): "'host' to stop on host packages.") parser.add_argument("--exclude", "-x", metavar="PACKAGE", dest="exclude_list", action="append", help="Like --stop-on, but do not add PACKAGE to the graph.") - parser.add_argument("--colours", "-c", metavar="COLOR_LIST", dest="colours", + parser.add_argument("--colors", "-c", metavar="COLOR_LIST", dest="colors", default="lightblue,grey,gainsboro", - help="Comma-separated list of the three colours to use" + + help="Comma-separated list of the three colors to use" + " to draw the top-level package, the target" + " packages, and the host packages, in this order." + " Defaults to: 'lightblue,grey,gainsboro'")
@@ -353,12 +353,12 @@ def main(): get_depends_func = brpkgutil.get_rdepends arrow_dir = "back" - # Get the colours: we need exactly three colours, + # Get the colors: we need exactly three colors, # so no need not split more than 4 - # We'll let 'dot' validate the colours... - colours = args.colours.split(',', 4) - if len(colours) != 3: - sys.stderr.write("Error: incorrect colour list '%s'\n" % args.colours) + # We'll let 'dot' validate the colors... + colors = args.colors.split(',', 4) + if len(colors) != 3: + sys.stderr.write("Error: incorrect color list '%s'\n" % args.colors) sys.exit(1) # In full mode, start with the result of get_targets() to get the main
@@ -403,7 +403,7 @@ def main(): outfile.write("digraph G {\n") print_pkg_deps(outfile, dict_deps, dict_version, stop_list, exclude_list, - arrow_dir, 0, args.depth, rootpkg, colours) + arrow_dir, 0, args.depth, rootpkg, colors) outfile.write("}\n")
Signature · 1 line
2.14.3
PATCH v2 2/5 support/scripts/graph-depends: use colors instead of colours
Thomas, All,
On 2018-03-31 18:35 +0200, Thomas Petazzoni spake thusly:
Quoted 4 lines
The graph-depends was not very consistent in colors vs. colours: some parts were using colours, some parts were using colors.
Let's settle on the US spelling, colors.
I would have gone with the superior, historical, better, cleaner, nicer, and correct englih spelling instead... ;-]
Quoted 6 lines
This change the user-visble option --colours to --colors, but it is unlikely that a lot of users customize the colors through
BR2_GRAPH_DEPS_OPTS, so this user interface change is considered reasonable.Signed-off-by Thomas Petazzoni <[email protected]>
Regards,
Yann E. MORIN.
Quoted 74 lines
--- docs/manual/common-usage.txt | 4 ++-- support/scripts/graph-depends | 16 ++++++++-------- 2 files changed, 10 insertions(+), 10 deletions(-)docs/manual/common-usage.txt +2
-2@@ -224,12 +224,12 @@ The +graph-depends+ behaviour can be controlled by setting options in the * +--transitive+, +--no-transitive+, to draw (or not) the transitive dependencies. The default is to not draw transitive dependencies. -* +--colours R,T,H+, the comma-separated list of colours to draw the +* +--colors R,T,H+, the comma-separated list of colors to draw the root package (+R+), the target packages (+T+) and the host packages (+H+). Defaults to: +lightblue,grey,gainsboro+ -------------------------------- -BR2_GRAPH_DEPS_OPTS='-d 3 --no-transitive --colours=red,green,blue' make graph-depends +BR2_GRAPH_DEPS_OPTS='-d 3 --no-transitive --colors=red,green,blue' make graph-depends -------------------------------- === Graphing the build durationsupport/scripts/graph-depends +8
-8@@ -297,9 +297,9 @@ def parse_args(): "'host' to stop on host packages.") parser.add_argument("--exclude", "-x", metavar="PACKAGE", dest="exclude_list", action="append", help="Like --stop-on, but do not add PACKAGE to the graph.") - parser.add_argument("--colours", "-c", metavar="COLOR_LIST", dest="colours", + parser.add_argument("--colors", "-c", metavar="COLOR_LIST", dest="colors", default="lightblue,grey,gainsboro", - help="Comma-separated list of the three colours to use" + + help="Comma-separated list of the three colors to use" + " to draw the top-level package, the target" + " packages, and the host packages, in this order." + " Defaults to: 'lightblue,grey,gainsboro'")@@ -353,12 +353,12 @@ def main(): get_depends_func = brpkgutil.get_rdepends arrow_dir = "back" - # Get the colours: we need exactly three colours, + # Get the colors: we need exactly three colors, # so no need not split more than 4 - # We'll let 'dot' validate the colours... - colours = args.colours.split(',', 4) - if len(colours) != 3: - sys.stderr.write("Error: incorrect colour list '%s'\n" % args.colours) + # We'll let 'dot' validate the colors... + colors = args.colors.split(',', 4) + if len(colors) != 3: + sys.stderr.write("Error: incorrect color list '%s'\n" % args.colors) sys.exit(1) # In full mode, start with the result of get_targets() to get the main@@ -403,7 +403,7 @@ def main(): outfile.write("digraph G {\n") print_pkg_deps(outfile, dict_deps, dict_version, stop_list, exclude_list, - arrow_dir, 0, args.depth, rootpkg, colours) + arrow_dir, 0, args.depth, rootpkg, colors) outfile.write("}\n")Signature · 6 lines
2.14.3 _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH v2 2/5 support/scripts/graph-depends: use colors instead of colours
In reply to Thomas PetazzoniQuoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Thomas" == Thomas Petazzoni <[email protected]> writes:
The graph-depends was not very consistent in colors vs. colours: some parts were using colours, some parts were using colors.
Quoted 8 lines
Let's settle on the US spelling, colors.
This change the user-visble option --colours to --colors, but it is unlikely that a lot of users customize the colors through
BR2_GRAPH_DEPS_OPTS, so this user interface change is considered reasonable.Signed-off-by Thomas Petazzoni <[email protected]>
Committed, thanks.
Signature · 1 line
Bye, Peter Korsgaard
PATCH v2 3/5 support/scripts/graph-depends: add --quiet option
In reply to Thomas PetazzoniThis will be useful for the upcoming recursive show-depends and show-rdepends features.
--- support/scripts/brpkgutil.py | 18 ++++++++++-------- support/scripts/graph-depends | 22 +++++++++++++--------- 2 files changed, 23 insertions(+), 17 deletions(-)
support/scripts/brpkgutil.py +10 -8
@@ -6,8 +6,9 @@ import subprocess # Execute the "make <pkg>-show-version" command to get the version of a given # list of packages, and return the version formatted as a Python dictionary. -def get_version(pkgs): - sys.stderr.write("Getting version for %s\n" % pkgs) +def get_version(pkgs, quiet=False): + if not quiet: + sys.stderr.write("Getting version for %s\n" % pkgs) cmd = ["make", "-s", "--no-print-directory"] for pkg in pkgs: cmd.append("%s-show-version" % pkg)
@@ -27,8 +28,9 @@ def get_version(pkgs): return version -def _get_depends(pkgs, rule): - sys.stderr.write("Getting dependencies for %s\n" % pkgs) +def _get_depends(pkgs, rule, quiet=False): + if not quiet: + sys.stderr.write("Getting dependencies for %s\n" % pkgs) cmd = ["make", "-s", "--no-print-directory"] for pkg in pkgs: cmd.append("%s-%s" % (pkg, rule))
@@ -55,12 +57,12 @@ def _get_depends(pkgs, rule): # Execute the "make <pkg>-show-depends" command to get the list of # dependencies of a given list of packages, and return the list of # dependencies formatted as a Python dictionary. -def get_depends(pkgs): - return _get_depends(pkgs, 'show-depends') +def get_depends(pkgs, quiet=False): + return _get_depends(pkgs, 'show-depends', quiet) # Execute the "make <pkg>-show-rdepends" command to get the list of # reverse dependencies of a given list of packages, and return the # list of dependencies formatted as a Python dictionary. -def get_rdepends(pkgs): - return _get_depends(pkgs, 'show-rdepends') +def get_rdepends(pkgs, quiet=False): + return _get_depends(pkgs, 'show-rdepends', quiet)
support/scripts/graph-depends +13 -9
@@ -38,8 +38,9 @@ allpkgs = [] # Execute the "make show-targets" command to get the list of the main # Buildroot PACKAGES and return it formatted as a Python list. This # list is used as the starting point for full dependency graphs -def get_targets(): - sys.stderr.write("Getting targets\n") +def get_targets(quiet): + if not quiet: + sys.stderr.write("Getting targets\n") cmd = ["make", "-s", "--no-print-directory", "show-targets"] p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True) output = p.communicate()[0].strip()
@@ -55,7 +56,7 @@ def get_targets(): # 'dependencies', which contains tuples of the form (pkg1 -> # pkg2_on_which_pkg1_depends, pkg3 -> pkg4_on_which_pkg3_depends) and # the function finally returns this list. -def get_all_depends(pkgs, get_depends_func): +def get_all_depends(pkgs, get_depends_func, quiet): dependencies = [] # Filter the packages for which we already have the dependencies
@@ -69,7 +70,7 @@ def get_all_depends(pkgs, get_depends_func): if len(filtered_pkgs) == 0: return [] - depends = get_depends_func(filtered_pkgs) + depends = get_depends_func(filtered_pkgs, quiet) deps = set() for pkg in filtered_pkgs:
@@ -85,7 +86,7 @@ def get_all_depends(pkgs, get_depends_func): deps.add(dep) if len(deps) != 0: - newdeps = get_all_depends(deps, get_depends_func) + newdeps = get_all_depends(deps, get_depends_func, quiet) if newdeps is not None: dependencies += newdeps
@@ -311,6 +312,8 @@ def parse_args(): help="Draw direct dependencies (the default)") parser.add_argument("--reverse", dest="direct", action='store_false', help="Draw reverse dependencies") + parser.add_argument("--quiet", '-q', dest="quiet", action='store_true', + help="Quiet") return parser.parse_args()
@@ -364,7 +367,7 @@ def main(): # In full mode, start with the result of get_targets() to get the main # targets and then use get_all_depends() for all targets if mode == MODE_FULL: - targets = get_targets() + targets = get_targets(args.quiet) dependencies = [] allpkgs.append('all') filtered_targets = []
@@ -374,7 +377,7 @@ def main(): continue dependencies.append(('all', tg)) filtered_targets.append(tg) - deps = get_all_depends(filtered_targets, get_depends_func) + deps = get_all_depends(filtered_targets, get_depends_func, args.quiet) if deps is not None: dependencies += deps rootpkg = 'all'
@@ -382,7 +385,7 @@ def main(): # In pkg mode, start directly with get_all_depends() on the requested # package elif mode == MODE_PKG: - dependencies = get_all_depends([rootpkg], get_depends_func) + dependencies = get_all_depends([rootpkg], get_depends_func, args.quiet) # Make the dependencies a dictionnary { 'pkg':[dep1, dep2, ...] } dict_deps = {}
@@ -397,7 +400,8 @@ def main(): dict_deps = remove_extra_deps(dict_deps, args.transitive) dict_version = brpkgutil.get_version([pkg for pkg in allpkgs - if pkg != "all" and not pkg.startswith("root")]) + if pkg != "all" and not pkg.startswith("root")], + args.quiet) # Start printing the graph data outfile.write("digraph G {\n")
Signature · 1 line
2.14.3
PATCH v2 3/5 support/scripts/graph-depends: add --quiet option
Thomas, All,
On 2018-03-31 18:35 +0200, Thomas Petazzoni spake thusly:
Quoted 14 lines
This will be useful for the upcoming recursive show-depends and show-rdepends features.
Signed-off-by Thomas Petazzoni <[email protected]>--- support/scripts/brpkgutil.py | 18 ++++++++++-------- support/scripts/graph-depends | 22 +++++++++++++--------- 2 files changed, 23 insertions(+), 17 deletions(-)support/scripts/brpkgutil.py +0
-0
support/scripts/brpkgutil.py import subprocess · line 11
Quoted 8 lines
support/scripts/brpkgutil.py +3
-2@@ -6,8 +6,9 @@ import subprocess # Execute the "make <pkg>-show-version" command to get the version of a given # list of packages, and return the version formatted as a Python dictionary. -def get_version(pkgs): - sys.stderr.write("Getting version for %s\n" % pkgs) +def get_version(pkgs, quiet=False): + if not quiet: + sys.stderr.write("Getting version for %s\n" % pkgs)
Having all those 'quiet' being propagated everywhere is ugly... ;-)
Instead, I would have done something like:
def verbose_trace(.....):
sys.stderr.write(....)
def quite_trace(......):
passand then in main:
if needs_traces:
trace = verbose_trace
else
trace = quite_trace
and then everywhere you need to trace, just call trace(...)
Regards,
Yann E. MORIN.
Quoted 128 lines
support/scripts/brpkgutil.py +7
-6cmd = ["make", "-s", "--no-print-directory"] for pkg in pkgs: cmd.append("%s-show-version" % pkg)@@ -27,8 +28,9 @@ def get_version(pkgs): return version -def _get_depends(pkgs, rule): - sys.stderr.write("Getting dependencies for %s\n" % pkgs) +def _get_depends(pkgs, rule, quiet=False): + if not quiet: + sys.stderr.write("Getting dependencies for %s\n" % pkgs) cmd = ["make", "-s", "--no-print-directory"] for pkg in pkgs: cmd.append("%s-%s" % (pkg, rule))@@ -55,12 +57,12 @@ def _get_depends(pkgs, rule): # Execute the "make <pkg>-show-depends" command to get the list of # dependencies of a given list of packages, and return the list of # dependencies formatted as a Python dictionary. -def get_depends(pkgs): - return _get_depends(pkgs, 'show-depends') +def get_depends(pkgs, quiet=False): + return _get_depends(pkgs, 'show-depends', quiet) # Execute the "make <pkg>-show-rdepends" command to get the list of # reverse dependencies of a given list of packages, and return the # list of dependencies formatted as a Python dictionary. -def get_rdepends(pkgs): - return _get_depends(pkgs, 'show-rdepends') +def get_rdepends(pkgs, quiet=False): + return _get_depends(pkgs, 'show-rdepends', quiet)support/scripts/graph-depends +13
-9@@ -38,8 +38,9 @@ allpkgs = [] # Execute the "make show-targets" command to get the list of the main # Buildroot PACKAGES and return it formatted as a Python list. This # list is used as the starting point for full dependency graphs -def get_targets(): - sys.stderr.write("Getting targets\n") +def get_targets(quiet): + if not quiet: + sys.stderr.write("Getting targets\n") cmd = ["make", "-s", "--no-print-directory", "show-targets"] p = subprocess.Popen(cmd, stdout=subprocess.PIPE, universal_newlines=True) output = p.communicate()[0].strip()@@ -55,7 +56,7 @@ def get_targets(): # 'dependencies', which contains tuples of the form (pkg1 -> # pkg2_on_which_pkg1_depends, pkg3 -> pkg4_on_which_pkg3_depends) and # the function finally returns this list. -def get_all_depends(pkgs, get_depends_func): +def get_all_depends(pkgs, get_depends_func, quiet): dependencies = [] # Filter the packages for which we already have the dependencies@@ -69,7 +70,7 @@ def get_all_depends(pkgs, get_depends_func): if len(filtered_pkgs) == 0: return [] - depends = get_depends_func(filtered_pkgs) + depends = get_depends_func(filtered_pkgs, quiet) deps = set() for pkg in filtered_pkgs:@@ -85,7 +86,7 @@ def get_all_depends(pkgs, get_depends_func): deps.add(dep) if len(deps) != 0: - newdeps = get_all_depends(deps, get_depends_func) + newdeps = get_all_depends(deps, get_depends_func, quiet) if newdeps is not None: dependencies += newdeps@@ -311,6 +312,8 @@ def parse_args(): help="Draw direct dependencies (the default)") parser.add_argument("--reverse", dest="direct", action='store_false', help="Draw reverse dependencies") + parser.add_argument("--quiet", '-q', dest="quiet", action='store_true', + help="Quiet") return parser.parse_args()@@ -364,7 +367,7 @@ def main(): # In full mode, start with the result of get_targets() to get the main # targets and then use get_all_depends() for all targets if mode == MODE_FULL: - targets = get_targets() + targets = get_targets(args.quiet) dependencies = [] allpkgs.append('all') filtered_targets = []@@ -374,7 +377,7 @@ def main(): continue dependencies.append(('all', tg)) filtered_targets.append(tg) - deps = get_all_depends(filtered_targets, get_depends_func) + deps = get_all_depends(filtered_targets, get_depends_func, args.quiet) if deps is not None: dependencies += deps rootpkg = 'all'@@ -382,7 +385,7 @@ def main(): # In pkg mode, start directly with get_all_depends() on the requested # package elif mode == MODE_PKG: - dependencies = get_all_depends([rootpkg], get_depends_func) + dependencies = get_all_depends([rootpkg], get_depends_func, args.quiet) # Make the dependencies a dictionnary { 'pkg':[dep1, dep2, ...] } dict_deps = {}@@ -397,7 +400,8 @@ def main(): dict_deps = remove_extra_deps(dict_deps, args.transitive) dict_version = brpkgutil.get_version([pkg for pkg in allpkgs - if pkg != "all" and not pkg.startswith("root")]) + if pkg != "all" and not pkg.startswith("root")], + args.quiet) # Start printing the graph data outfile.write("digraph G {\n")Signature · 6 lines
2.14.3 _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH v2 4/5 support/scripts/graph-depends: add --flat-list option
In reply to Thomas PetazzoniFrom: George Redivo <[email protected]>
graph-depends currently spits out a graph in .dot format. However, as part of the upcoming introduction of <pkg>-show-recursive-depends and <pkg>-show-recursive-rdepends, we need graph-depends to be able to display a flat list.
[Thomas:
- Rebase on top of graph-depends changes
- Do not display the package name itself in the list, only its dependencies (or reverse dependencies)
- Display the result on a single line, instead of one package per line, in order to match what <pkg>-show-depends and <pkg>-show-rdepends are doing today.]
--- support/scripts/graph-depends | 26 +++++++++++++++++++------- 1 file changed, 19 insertions(+), 7 deletions(-)
support/scripts/graph-depends +19 -7
@@ -248,11 +248,14 @@ done_deps = [] # Print the dependency graph of a package def print_pkg_deps(outfile, dict_deps, dict_version, stop_list, exclude_list, - arrow_dir, depth, max_depth, pkg, colors): + arrow_dir, draw_graph, depth, max_depth, pkg, colors): if pkg in done_deps: return done_deps.append(pkg) - print_attrs(outfile, pkg, dict_version.get(pkg), depth, colors) + if draw_graph: + print_attrs(outfile, pkg, dict_version.get(pkg), depth, colors) + elif depth != 0: + outfile.write("%s " % pkg) if pkg not in dict_deps: return for p in stop_list:
@@ -276,9 +279,10 @@ def print_pkg_deps(outfile, dict_deps, dict_version, stop_list, exclude_list, add = False break if add: - outfile.write("%s -> %s [dir=%s]\n" % (pkg_node_name(pkg), pkg_node_name(d), arrow_dir)) + if draw_graph: + outfile.write("%s -> %s [dir=%s]\n" % (pkg_node_name(pkg), pkg_node_name(d), arrow_dir)) print_pkg_deps(outfile, dict_deps, dict_version, stop_list, exclude_list, - arrow_dir, depth + 1, max_depth, d, colors) + arrow_dir, draw_graph, depth + 1, max_depth, d, colors) def parse_args():
@@ -314,6 +318,8 @@ def parse_args(): help="Draw reverse dependencies") parser.add_argument("--quiet", '-q', dest="quiet", action='store_true', help="Quiet") + parser.add_argument("--flat-list", '-f', dest="flat_list", action='store_true', default=False, + help="Do not draw graph, just print a flat list") return parser.parse_args()
@@ -356,6 +362,8 @@ def main(): get_depends_func = brpkgutil.get_rdepends arrow_dir = "back" + draw_graph = not args.flat_list + # Get the colors: we need exactly three colors, # so no need not split more than 4 # We'll let 'dot' validate the colors...
@@ -404,12 +412,16 @@ def main(): args.quiet) # Start printing the graph data - outfile.write("digraph G {\n") + if draw_graph: + outfile.write("digraph G {\n") print_pkg_deps(outfile, dict_deps, dict_version, stop_list, exclude_list, - arrow_dir, 0, args.depth, rootpkg, colors) + arrow_dir, draw_graph, 0, args.depth, rootpkg, colors) - outfile.write("}\n") + if draw_graph: + outfile.write("}\n") + else: + outfile.write("\n") if __name__ == "__main__":
Signature · 1 line
2.14.3
PATCH v2 4/5 support/scripts/graph-depends: add --flat-list option
Quoted 9 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Thomas" == Thomas Petazzoni <[email protected]> writes:
From: George Redivo <[email protected]> graph-depends currently spits out a graph in .dot format. However, as part of the upcoming introduction of <pkg>-show-recursive-depends and <pkg>-show-recursive-rdepends, we need graph-depends to be able to display a flat list.
Signed-off-by George Redivo <[email protected]>
Quoted 8 lines
[Thomas:
- Rebase on top of graph-depends changes
- Do not display the package name itself in the list, only its dependencies (or reverse dependencies)
- Display the result on a single line, instead of one package per line, in order to match what <pkg>-show-depends and <pkg>-show-rdepends are doing today.]Signed-off-by Thomas Petazzoni <[email protected]>
Committed, thanks.
Signature · 1 line
Bye, Peter Korsgaard
PATCH v2 5/5 package/pkg-generic: add <pkg>-show-recursive-(r)depends targets
In reply to Thomas PetazzoniFrom: George Redivo <[email protected]>
This commit adds the support for <pkg>-show-recursive-depends and <pkg>-show-recursive-rdepends which respectively show the list of all dependencies or reverse dependencies for a given package. The existing show-depends and show-rdepends only show the first-level dependencies, while show-recursive-depends and show-recursive-rdepends show recursively the dependencies.
It is worth mentioning that while show-recursive-depends really shows all dependencies, show-recursive-rdepends is a bit limited because the reverse dependencies of host packages are not properly accounted for. But that's a limitation that already exists in show-rdepends, and that cannot easily be solved.
[Thomas:
- split from the patch that was also changing graph-depends
- rename show-rrdepends to show-recursive-rdepends
- add show-recursive-depends
- don't create GRAPHS_DIR.]
--- Makefile | 4 ++++ package/pkg-generic.mk | 8 ++++++++ 2 files changed, 12 insertions(+)
Makefile +4 -0
@@ -1024,6 +1024,10 @@ help: @echo ' <pkg>-build - Build <pkg> up to the build step' @echo ' <pkg>-show-depends - List packages on which <pkg> depends' @echo ' <pkg>-show-rdepends - List packages which have <pkg> as a dependency' + @echo ' <pkg>-show-recursive-depends' + @echo ' - Recursively list packages on which <pkg> depends' + @echo ' <pkg>-show-recursive-rdepends' + @echo ' - Recursively list packages which have <pkg> as a dependency' @echo ' <pkg>-graph-depends - Generate a graph of <pkg>'\''s dependencies' @echo ' <pkg>-graph-rdepends - Generate a graph of <pkg>'\''s reverse dependencies' @echo ' <pkg>-dirclean - Remove <pkg> build directory'
package/pkg-generic.mk +8 -0
@@ -759,9 +759,17 @@ $(1)-show-version: $(1)-show-depends: @echo $$($(2)_FINAL_ALL_DEPENDENCIES) +$(1)-show-recursive-depends: + @cd "$$(CONFIG_DIR)" && \ + $$(TOPDIR)/support/scripts/graph-depends -p $(1) -f -q + $(1)-show-rdepends: @echo $$($(2)_RDEPENDENCIES) +$(1)-show-recursive-rdepends: + @cd "$$(CONFIG_DIR)" && \ + $$(TOPDIR)/support/scripts/graph-depends -p $(1) --reverse -f -q + $(1)-show-build-order: $$(patsubst %,%-show-build-order,$$($(2)_FINAL_ALL_DEPENDENCIES)) $$(info $(1))
Signature · 1 line
2.14.3
PATCH v2 5/5 package/pkg-generic: add <pkg>-show-recursive-(r)depends targets
Quoted 16 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Thomas" == Thomas Petazzoni <[email protected]> writes:
From: George Redivo <[email protected]>
This commit adds the support for <pkg>-show-recursive-depends and <pkg>-show-recursive-rdepends which respectively show the list of all dependencies or reverse dependencies for a given package. The existing show-depends and show-rdepends only show the first-level dependencies, while show-recursive-depends and show-recursive-rdepends show recursively the dependencies.It is worth mentioning that while show-recursive-depends really shows all dependencies, show-recursive-rdepends is a bit limited because the reverse dependencies of host packages are not properly accounted for. But that's a limitation that already exists in show-rdepends, and that cannot easily be solved.
Quoted 7 lines
Signed-off-by George Redivo <[email protected]>[Thomas:
- split from the patch that was also changing graph-depends
- rename show-rrdepends to show-recursive-rdepends
- add show-recursive-depends
- don't createGRAPHS_DIR.]Signed-off-by Thomas Petazzoni <[email protected]>
Committed, thanks.
Signature · 1 line
Bye, Peter Korsgaard