support/scripts: use the new show-info to generate dependency graphs (branch yem/show-info-2)
v3 cover support/scripts: use the new show-info to generate dependency graphs (branch yem/show-info-2)
Hello All!
The dependency graphs currently use the show-depednency-tree rule, which was introduced in this release cycle. But we now have a better and more versatile rule, show-info, to provide the same (and more) information.
So, this series switches brpkg-utils to use show-info to grab the dependency tree, and then removes the show-dependency-tree rule as it was never part of a release.
Regards,
Yann E. MORIN.
The following changes since commit ce32fa6cc1decd07ced9f944e7ea6aa51dd0c49d
package/go: bump version to 1.12.5 (2019-05-07 22:30:13 +0200)
are available in the git repository at:
git://git.buildroot.org/~ymorin/git/buildroot.git
for you to fetch changes up to fc3002077beb3f48401c4280e5347f27f548b4a3
core: remove show-dependency-tree (2019-05-07 22:51:00 +0200)
----------------------------------------------------------------
Yann E. MORIN (2):
support/scripts: use show-info to extract dependency graph
core: remove show-dependency-tree
Makefile | 4 ---- fs/common.mk | 8 -------- package/pkg-generic.mk | 6 ------ support/scripts/brpkgutil.py | 29 ++++++++++++++--------------- 4 files changed, 14 insertions(+), 33 deletions(-)
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 561 099 427 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH v3 1/2 support/scripts: use show-info to extract dependency graph
Currently, we extract the dependency graph from the aptly named but ad-hoc show-dependency-graph rule.
We now have a better solution to report package information, with show-info.
Since show-dependency-graph never went into a release so far, and show-info does provide the same (and more), switch to using show-info.
Thanks to Adam for suggesting the coding style to have a readable code that is not ugly but still pleases flake8. Thanks to Arnout for suggesting the use of dict.get() to further simplify the code.
Note: we do not use the reverse_dependencies field because it only contains those packages that have a kconfig option, so we'd miss most host packages.
--- support/scripts/brpkgutil.py | 29 ++++++++++++++--------------- 1 file changed, 14 insertions(+), 15 deletions(-)
support/scripts/brpkgutil.py +14 -15
@@ -1,6 +1,7 @@ # Copyright (C) 2010-2013 Thomas Petazzoni <[email protected]> # Copyright (C) 2019 Yann E. MORIN <[email protected]> +import json import logging import os import subprocess
@@ -19,7 +20,7 @@ from collections import defaultdict def get_dependency_tree(): logging.info("Getting dependency tree...") - deps = defaultdict(list) + deps = {} rdeps = defaultdict(list) types = {} versions = {}
@@ -29,23 +30,21 @@ def get_dependency_tree(): types['all'] = 'target' versions['all'] = '' - cmd = ["make", "-s", "--no-print-directory", "show-dependency-tree"] + cmd = ["make", "-s", "--no-print-directory", "show-info"] with open(os.devnull, 'wb') as devnull: p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=devnull, universal_newlines=True) - output = p.communicate()[0] + pkg_list = json.loads(p.communicate()[0]) - for l in output.splitlines(): - if " -> " in l: - pkg = l.split(" -> ")[0] - deps[pkg] += l.split(" -> ")[1].split() - for p in l.split(" -> ")[1].split(): - rdeps[p].append(pkg) - else: - pkg, type_version = l.split(": ", 1) - t, v = "{} -".format(type_version).split(None, 2)[:2] - deps['all'].append(pkg) - types[pkg] = t - versions[pkg] = v + for pkg in pkg_list: + deps['all'].append(pkg) + types[pkg] = pkg_list[pkg]["type"] + deps[pkg] = pkg_list[pkg].get("dependencies", []) + for p in deps[pkg]: + rdeps[p].append(pkg) + versions[pkg] = \ + None if pkg_list[pkg]["type"] == "rootfs" \ + else "virtual" if pkg_list[pkg]["virtual"] \ + else pkg_list[pkg]["version"] return (deps, rdeps, types, versions)
Signature · 1 line
2.20.1
PATCH v3 1/2 support/scripts: use show-info to extract dependency graph
Quoted 19 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Currently, we extract the dependency graph from the aptly named but ad-hoc show-dependency-graph rule.
We now have a better solution to report package information, with show-info.
Since show-dependency-graph never went into a release so far, and show-info does provide the same (and more), switch to using show-info.
Thanks to Adam for suggesting the coding style to have a readable code that is not ugly but still pleases flake8. Thanks to Arnout for suggesting the use of
dict.get()to further simplify the code.Note: we do not use the
reverse_dependenciesfield because it only contains those packages that have a kconfig option, so we'd miss most host packages.
Quoted 7 lines
Reported-by Thomas Petazzoni <[email protected]>Signed-off-by "Yann E. MORIN" <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Thomas De Schampheleire <[email protected]>Cc Adam Duskett <[email protected]>Cc Arnout Vandecappelle <[email protected]>Cc Ricardo Martincoski <[email protected]>
Committed, thanks.
Signature · 1 line
Bye, Peter Korsgaard
PATCH v3 2/2 core: remove show-dependency-tree
In reply to Yann E. MORINshow-dependency-tree was introduced in this release cycle, as a way to quickly and easily provide the dependency tree to graph-depends.
show-dependency-tree is no longer used, now that graph-depends has been switched over to using the more versatile show-info.
Beside, show-dependency-tree has never been part of a release.
Drop it.
--- Makefile | 4 ---- fs/common.mk | 8 -------- package/pkg-generic.mk | 6 ------ 3 files changed, 18 deletions(-)
Makefile +0 -4
@@ -877,10 +877,6 @@ graph-depends-requirements: @dot -? >/dev/null 2>&1 || \ { echo "ERROR: The 'dot' program from Graphviz is needed for graph-depends" >&2; exit 1; } -.PHONY: show-dependency-tree -show-dependency-tree: $(patsubst %,%-show-dependency-tree,$(PACKAGES) $(TARGETS_ROOTFS)) - @: - .PHONY: graph-depends graph-depends: graph-depends-requirements @$(INSTALL) -d $(GRAPHS_DIR)
fs/common.mk +0 -8
@@ -61,10 +61,6 @@ ROOTFS_COMMON_FINAL_RECURSIVE_DEPENDENCIES = $(sort \ ) \ $(ROOTFS_COMMON_FINAL_RECURSIVE_DEPENDENCIES__X)) -rootfs-common-show-dependency-tree: $(patsubst %,%-show-dependency-tree,$(ROOTFS_COMMON_DEPENDENCIES)) - $(info rootfs-common: host) - $(info rootfs-common -> $(foreach d,$(ROOTFS_COMMON_DEPENDENCIES),$(d))) - .PHONY: rootfs-common rootfs-common: $(ROOTFS_COMMON_DEPENDENCIES) target-finalize @$(call MESSAGE,"Generating root filesystems common tables")
@@ -117,10 +113,6 @@ ROOTFS_$(2)_FINAL_RECURSIVE_DEPENDENCIES = $$(sort \ ) \ $$(ROOTFS_$(2)_FINAL_RECURSIVE_DEPENDENCIES__X)) -rootfs-$(1)-show-dependency-tree: $$(patsubst %,%-show-dependency-tree,$$(ROOTFS_$(2)_DEPENDENCIES)) - $$(info rootfs-$(1): host) - $$(info rootfs-$(1) -> $$(foreach d,$$(ROOTFS_$(2)_DEPENDENCIES),$$(d))) - ifeq ($$(BR2_TARGET_ROOTFS_$(2)_GZIP),y) ROOTFS_$(2)_COMPRESS_EXT = .gz ROOTFS_$(2)_COMPRESS_CMD = gzip -9 -c
package/pkg-generic.mk +0 -6
@@ -862,11 +862,6 @@ $(1)-show-build-order: $$(patsubst %,%-show-build-order,$$($(2)_FINAL_ALL_DEPEND @: $$(info $(1)) -$(1)-show-dependency-tree: $$(patsubst %,%-show-dependency-tree,$$($(2)_FINAL_ALL_DEPENDENCIES)) - @: - $$(info $(1): $(4) $$(if $$($(2)_IS_VIRTUAL),virtual,$$($(2)_DL_VERSION))) - $$(info $(1) -> $$($(2)_FINAL_ALL_DEPENDENCIES)) - $(1)-show-info: @: $$(info $$(call clean-json,{ $$(call json-info,$(2)) }))
@@ -1101,7 +1096,6 @@ DL_TOOLS_DEPENDENCIES += $$(call extractor-dependency,$$($(2)_SOURCE)) $(1)-reconfigure \ $(1)-reinstall \ $(1)-rsync \ - $(1)-show-dependency-tree \ $(1)-show-depends \ $(1)-show-info \ $(1)-show-version \
Signature · 1 line
2.20.1
PATCH v3 2/2 core: remove show-dependency-tree
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
show-dependency-tree was introduced in this release cycle, as a way to quickly and easily provide the dependency tree to graph-depends.
show-dependency-tree is no longer used, now that graph-depends has been switched over to using the more versatile show-info.
Quoted 8 lines
Beside, show-dependency-tree has never been part of a release.
Drop it.
Signed-off-by "Yann E. MORIN" <[email protected]>Cc Thomas Petazzoni <[email protected]>Cc Thomas De Schampheleire <[email protected]>Cc Arnout Vandecappelle <[email protected]>
Committed, thanks.
Signature · 1 line
Bye, Peter Korsgaard