Skip to contentSkip to conversation

support/scripts: use the new show-info to generate dependency graphs (branch yem/show-info-2)

PATCH New Yann E. MORIN 2 participants 5 messages May 7, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v3 cover support/scripts: use the new show-info to generate dependency graphs (branch yem/show-info-2)

Hello All!

The dependency graphs currently use the show-depednency-tree rule, which was introduced in this release cycle. But we now have a better and more versatile rule, show-info, to provide the same (and more) information.

So, this series switches brpkg-utils to use show-info to grab the dependency tree, and then removes the show-dependency-tree rule as it was never part of a release.

Regards,
Yann E. MORIN.

The following changes since commit ce32fa6cc1decd07ced9f944e7ea6aa51dd0c49d

package/go: bump version to 1.12.5 (2019-05-07 22:30:13 +0200)

are available in the git repository at:

git://git.buildroot.org/~ymorin/git/buildroot.git

for you to fetch changes up to fc3002077beb3f48401c4280e5347f27f548b4a3

core: remove show-dependency-tree (2019-05-07 22:51:00 +0200)

----------------------------------------------------------------
Yann E. MORIN (2):
support/scripts: use show-info to extract dependency graph
core: remove show-dependency-tree

 Makefile                     |  4 ----
 fs/common.mk                 |  8 --------
 package/pkg-generic.mk       |  6 ------
 support/scripts/brpkgutil.py | 29 ++++++++++++++---------------
 4 files changed, 14 insertions(+), 33 deletions(-)
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
#1 lore raw

PATCH v3 1/2 support/scripts: use show-info to extract dependency graph

Currently, we extract the dependency graph from the aptly named but ad-hoc show-dependency-graph rule.

We now have a better solution to report package information, with show-info.

Since show-dependency-graph never went into a release so far, and show-info does provide the same (and more), switch to using show-info.

Thanks to Adam for suggesting the coding style to have a readable code that is not ugly but still pleases flake8. Thanks to Arnout for suggesting the use of dict.get() to further simplify the code.

Note: we do not use the reverse_dependencies field because it only contains those packages that have a kconfig option, so we'd miss most host packages.

Cc Thomas Petazzoni <[email protected]>
Cc Thomas De Schampheleire <[email protected]>
Cc Adam Duskett <[email protected]>
Cc Arnout Vandecappelle <[email protected]>
Cc Ricardo Martincoski <[email protected]>
---
 support/scripts/brpkgutil.py | 29 ++++++++++++++---------------
 1 file changed, 14 insertions(+), 15 deletions(-)
support/scripts/brpkgutil.py +14 -15
index f65068d348..73d8fbe931 100644
@@ -1,6 +1,7 @@
 # Copyright (C) 2010-2013 Thomas Petazzoni <[email protected]>
 # Copyright (C) 2019 Yann E. MORIN <[email protected]>
 
+import json
 import logging
 import os
 import subprocess
@@ -19,7 +20,7 @@ from collections import defaultdict
 def get_dependency_tree():
     logging.info("Getting dependency tree...")
 
-    deps = defaultdict(list)
+    deps = {}
     rdeps = defaultdict(list)
     types = {}
     versions = {}
@@ -29,23 +30,21 @@ def get_dependency_tree():
     types['all'] = 'target'
     versions['all'] = ''
 
-    cmd = ["make", "-s", "--no-print-directory", "show-dependency-tree"]
+    cmd = ["make", "-s", "--no-print-directory", "show-info"]
     with open(os.devnull, 'wb') as devnull:
         p = subprocess.Popen(cmd, stdout=subprocess.PIPE, stderr=devnull,
                              universal_newlines=True)
-        output = p.communicate()[0]
+        pkg_list = json.loads(p.communicate()[0])
 
-    for l in output.splitlines():
-        if " -> " in l:
-            pkg = l.split(" -> ")[0]
-            deps[pkg] += l.split(" -> ")[1].split()
-            for p in l.split(" -> ")[1].split():
-                rdeps[p].append(pkg)
-        else:
-            pkg, type_version = l.split(": ", 1)
-            t, v = "{} -".format(type_version).split(None, 2)[:2]
-            deps['all'].append(pkg)
-            types[pkg] = t
-            versions[pkg] = v
+    for pkg in pkg_list:
+        deps['all'].append(pkg)
+        types[pkg] = pkg_list[pkg]["type"]
+        deps[pkg] = pkg_list[pkg].get("dependencies", [])
+        for p in deps[pkg]:
+            rdeps[p].append(pkg)
+        versions[pkg] = \
+            None if pkg_list[pkg]["type"] == "rootfs" \
+            else "virtual" if pkg_list[pkg]["virtual"] \
+            else pkg_list[pkg]["version"]
 
     return (deps, rdeps, types, versions)
Signature · 1 line
2.20.1
#2 lore raw

PATCH v3 1/2 support/scripts: use show-info to extract dependency graph

Quoted 19 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

Currently, we extract the dependency graph from the aptly named but ad-hoc show-dependency-graph rule.

We now have a better solution to report package information, with show-info.

Since show-dependency-graph never went into a release so far, and show-info does provide the same (and more), switch to using show-info.

Thanks to Adam for suggesting the coding style to have a readable code that is not ugly but still pleases flake8. Thanks to Arnout for suggesting the use of dict.get() to further simplify the code.

Note: we do not use the reverse_dependencies field because it only contains those packages that have a kconfig option, so we'd miss most host packages.

Quoted 7 lines
Cc Thomas Petazzoni <[email protected]>
Cc Thomas De Schampheleire <[email protected]>
Cc Adam Duskett <[email protected]>
Cc Arnout Vandecappelle <[email protected]>
Cc Ricardo Martincoski <[email protected]>

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard
#3 lore raw

PATCH v3 2/2 core: remove show-dependency-tree

In reply to Yann E. MORIN

show-dependency-tree was introduced in this release cycle, as a way to quickly and easily provide the dependency tree to graph-depends.

show-dependency-tree is no longer used, now that graph-depends has been switched over to using the more versatile show-info.

Beside, show-dependency-tree has never been part of a release.

Drop it.

Cc Thomas Petazzoni <[email protected]>
Cc Thomas De Schampheleire <[email protected]>
Cc Arnout Vandecappelle <[email protected]>
---
 Makefile               | 4 ----
 fs/common.mk           | 8 --------
 package/pkg-generic.mk | 6 ------
 3 files changed, 18 deletions(-)
Makefile +0 -4
index 86e444da98..4da2453288 100644
@@ -877,10 +877,6 @@ graph-depends-requirements:
 	@dot -? >/dev/null 2>&1 || \
 		{ echo "ERROR: The 'dot' program from Graphviz is needed for graph-depends" >&2; exit 1; }
 
-.PHONY: show-dependency-tree
-show-dependency-tree: $(patsubst %,%-show-dependency-tree,$(PACKAGES) $(TARGETS_ROOTFS))
-	@:
-
 .PHONY: graph-depends
 graph-depends: graph-depends-requirements
 	@$(INSTALL) -d $(GRAPHS_DIR)
fs/common.mk +0 -8
index 6dd1774726..fceb38c5b9 100644
@@ -61,10 +61,6 @@ ROOTFS_COMMON_FINAL_RECURSIVE_DEPENDENCIES = $(sort \
 	) \
 	$(ROOTFS_COMMON_FINAL_RECURSIVE_DEPENDENCIES__X))
 
-rootfs-common-show-dependency-tree: $(patsubst %,%-show-dependency-tree,$(ROOTFS_COMMON_DEPENDENCIES))
-	$(info rootfs-common: host)
-	$(info rootfs-common -> $(foreach d,$(ROOTFS_COMMON_DEPENDENCIES),$(d)))
-
 .PHONY: rootfs-common
 rootfs-common: $(ROOTFS_COMMON_DEPENDENCIES) target-finalize
 	@$(call MESSAGE,"Generating root filesystems common tables")
@@ -117,10 +113,6 @@ ROOTFS_$(2)_FINAL_RECURSIVE_DEPENDENCIES = $$(sort \
 	) \
 	$$(ROOTFS_$(2)_FINAL_RECURSIVE_DEPENDENCIES__X))
 
-rootfs-$(1)-show-dependency-tree: $$(patsubst %,%-show-dependency-tree,$$(ROOTFS_$(2)_DEPENDENCIES))
-	$$(info rootfs-$(1): host)
-	$$(info rootfs-$(1) -> $$(foreach d,$$(ROOTFS_$(2)_DEPENDENCIES),$$(d)))
-
 ifeq ($$(BR2_TARGET_ROOTFS_$(2)_GZIP),y)
 ROOTFS_$(2)_COMPRESS_EXT = .gz
 ROOTFS_$(2)_COMPRESS_CMD = gzip -9 -c
package/pkg-generic.mk +0 -6
index 67782138b4..e44f5d913c 100644
@@ -862,11 +862,6 @@ $(1)-show-build-order: $$(patsubst %,%-show-build-order,$$($(2)_FINAL_ALL_DEPEND
 	@:
 	$$(info $(1))
 
-$(1)-show-dependency-tree: $$(patsubst %,%-show-dependency-tree,$$($(2)_FINAL_ALL_DEPENDENCIES))
-	@:
-	$$(info $(1): $(4) $$(if $$($(2)_IS_VIRTUAL),virtual,$$($(2)_DL_VERSION)))
-	$$(info $(1) -> $$($(2)_FINAL_ALL_DEPENDENCIES))
-
 $(1)-show-info:
 	@:
 	$$(info $$(call clean-json,{ $$(call json-info,$(2)) }))
@@ -1101,7 +1096,6 @@ DL_TOOLS_DEPENDENCIES += $$(call extractor-dependency,$$($(2)_SOURCE))
 	$(1)-reconfigure \
 	$(1)-reinstall \
 	$(1)-rsync \
-	$(1)-show-dependency-tree \
 	$(1)-show-depends \
 	$(1)-show-info \
 	$(1)-show-version \
Signature · 1 line
2.20.1
#4 lore raw

PATCH v3 2/2 core: remove show-dependency-tree

Quoted 8 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

"Yann" == Yann E MORIN <[email protected]> writes:

show-dependency-tree was introduced in this release cycle, as a way to quickly and easily provide the dependency tree to graph-depends.

show-dependency-tree is no longer used, now that graph-depends has been switched over to using the more versatile show-info.

Quoted 8 lines

Beside, show-dependency-tree has never been part of a release.

Drop it.

Cc Thomas Petazzoni <[email protected]>
Cc Thomas De Schampheleire <[email protected]>
Cc Arnout Vandecappelle <[email protected]>

Committed, thanks.

Signature · 1 line
Bye, Peter Korsgaard