graphs: add option to remove transitive dependencies in dependency graph
PATCH 1/1 graphs: add option to remove transitive dependencies in dependency graph
From: "Yann E. MORIN" <[email protected]>
Currently, all the dependencies of a package are drawn on the dependency graph, including transitive dependencies (e.g. A->B->C and A->C).
Very big graphs, with lots of packages with lots of dependencies, the dependency graph can be very dense, and transitive dependencies are cluttering the graph.
In some cases, only getting the "build-order" dependencies is enough (e.g. to see what impact a package rebuild would have).
Add a new environment variable to disable drawing transitive dependencies.
Basically, it would turn this graph:
pkg1 ---> pkg2 ---> pkg3 -------------------.
|\__________/ \ \
|\____________________ \ \
| \ \ \
`-> pkg4 ---> pkg5 ---> pkg6 ---> pkg7 ---> pkg8
\__________/into that graph:
pkg1 ---> pkg2 ---> pkg3 -----------.
| \
`-> pkg4 ---> pkg5 ---> pkg6 ---> pkg7 ---> pkg8---
Note: some graphs showing the results are there:
http://ymorin.is-a-geek.org/download/tmp/graphs/
--- Makefile | 9 +++++++-- package/pkg-generic.mk | 2 +- support/scripts/graph-depends | 45 +++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 53 insertions(+), 3 deletions(-)
Makefile +7 -2
@@ -150,7 +150,12 @@ endif # Need that early, before we scan packages # Avoids doing the $(or...) everytime BR_GRAPH_OUT := $(or $(BR2_GRAPH_OUT),pdf) -BR_GRAPH_DEPTH := $(or $(BR2_GRAPH_DEPTH),0) +BR_GRAPH_DEPS_OPTS := --depth $(or $(BR2_GRAPH_DEPTH),0) +ifeq ($(BR2_GRAPH_NO_TRANSITIVE),) +BR_GRAPH_DEPS_OPTS += --transitive +else +BR_GRAPH_DEPS_OPTS += --no-transitive +endif BUILD_DIR := $(BASE_DIR)/build BINARIES_DIR := $(BASE_DIR)/images
@@ -674,7 +679,7 @@ graph-build: $(O)/build/build-time.log graph-depends: @$(INSTALL) -d $(O)/graphs @cd "$(CONFIG_DIR)"; \ - $(TOPDIR)/support/scripts/graph-depends -d $(BR_GRAPH_DEPTH) \ + $(TOPDIR)/support/scripts/graph-depends $(BR_GRAPH_DEPS_OPTS) \ |tee $(O)/graphs/$(@).dot \ |dot -T$(BR_GRAPH_OUT) -o $(O)/graphs/$(@).$(BR_GRAPH_OUT)
package/pkg-generic.mk +1 -1
@@ -495,7 +495,7 @@ $(1)-show-depends: $(1)-graph-depends: @$(INSTALL) -d $(O)/graphs @cd "$(CONFIG_DIR)"; \ - $(TOPDIR)/support/scripts/graph-depends -p $(1) -d $(BR_GRAPH_DEPTH) \ + $(TOPDIR)/support/scripts/graph-depends -p $(1) $(BR_GRAPH_DEPS_OPTS) \ |tee $(O)/graphs/$$(@).dot \ |dot -T$(BR_GRAPH_OUT) -o $(O)/graphs/$$(@).$(BR_GRAPH_OUT)
support/scripts/graph-depends +45 -0
63 lines not on this page · raw message
Signature · 1 line
1.8.3.2
PATCH 1/1 graphs: add option to remove transitive dependencies in dependency graph
All,
On 2014-05-07 00:44 +0200, Yann E. MORIN spake thusly:
Quoted 5 lines
From: "Yann E. MORIN" <[email protected]>
Currently, all the dependencies of a package are drawn on the dependency graph, including transitive dependencies (e.g.
A->B->CandA->C).
Quoted 8 lines
Very big graphs, with lots of packages with lots of dependencies, the dependency graph can be very dense, and transitive dependencies are cluttering the graph.
In some cases, only getting the "build-order" dependencies is enough (e.g. to see what impact a package rebuild would have).
Add a new environment variable to disable drawing transitive dependencies.
I've marked it as Changes Requested in PAtchwork, since I have a cleaner patch to come soon, with other cleanups in the graph-depends script.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/1 graphs: add option to remove transitive dependencies in dependency graph
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
Hi,
Quoted 11 lines
All,
On 2014-05-07 00:44 +0200, Yann E. MORIN spake thusly:Quoted 9 lines
From: "Yann E. MORIN" <[email protected]>
Currently, all the dependencies of a package are drawn on the dependency graph, including transitive dependencies (e.g.
A->B->CandA->C).Very big graphs, with lots of packages with lots of dependencies, the dependency graph can be very dense, and transitive dependencies are cluttering the graph.
Quoted 7 lines
Quoted 4 lines
In some cases, only getting the "build-order" dependencies is enough (e.g. to see what impact a package rebuild would have).
Add a new environment variable to disable drawing transitive dependencies.
I've marked it as Changes Requested in PAtchwork, since I have a cleaner patch to come soon, with other cleanups in the graph-depends script.
Thanks. Like I mentioned on IRC I do like the looks of the graphs, but I'm not really happy with the use of environment variables to control it, as it isn't intuitive.
Perhaps we should simply have kconfig options for these things?
Signature · 1 line
Bye, Peter Korsgaard
PATCH 1/1 graphs: add option to remove transitive dependencies in dependency graph
Peter, All,
On 2014-05-09 12:00 +0200, Peter Korsgaard spake thusly:
Quoted 16 lines
Quoted 15 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
On 2014-05-07 00:44 +0200, Yann E. MORIN spake thusly:
Quoted 13 lines
From: "Yann E. MORIN" <[email protected]>
Currently, all the dependencies of a package are drawn on the dependency graph, including transitive dependencies (e.g.
A->B->CandA->C).Very big graphs, with lots of packages with lots of dependencies, the dependency graph can be very dense, and transitive dependencies are cluttering the graph.
In some cases, only getting the "build-order" dependencies is enough (e.g. to see what impact a package rebuild would have).
Add a new environment variable to disable drawing transitive dependencies.
Quoted 8 lines
Quoted 2 lines
I've marked it as Changes Requested in PAtchwork, since I have a cleaner patch to come soon, with other cleanups in the graph-depends script.
Thanks. Like I mentioned on IRC I do like the looks of the graphs, but I'm not really happy with the use of environment variables to control it, as it isn't intuitive.
Perhaps we should simply have kconfig options for these things?
As I said on IRC, I doubt this would be usefull, For example:
- for depth: someone might want a 3-level deep graph for some package, and a 4-level deep graph for another,
- for transitive deps: someone would like the complete dependencies graphed for some package, and only the 'build-order' deps for some another.
This means the user would have to go back to the menuconfig between each graph, and this could not be made automatic (eg. called from within a script that generates the graphs for the documentation of the project.)
What we could do, however, would be something like:
make BR2_GRAPH_OPTS='--depth 4 --no-transitive' graph-depends
(the existing BR2_GRPAH_DEPS would disapear).
Thoughts?
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/1 graphs: add option to remove transitive dependencies in dependency graph
Hi Yann, Peter, all
Quoted 25 lines · Yann E. MORIN
Peter, All,
On 2014-05-09 12:00 +0200, Peter Korsgaard spake thusly:
Quoted 22 lines
Quoted 11 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
"Yann" == Yann E MORIN <[email protected]> writes:
On 2014-05-07 00:44 +0200, Yann E. MORIN spake thusly:
Quoted 9 lines
From: "Yann E. MORIN" <[email protected]>
Currently, all the dependencies of a package are drawn on the dependency graph, including transitive dependencies (e.g.
A->B->CandA->C).Very big graphs, with lots of packages with lots of dependencies, the dependency graph can be very dense, and transitive dependencies are cluttering the graph.
Quoted 7 lines
Quoted 4 lines
In some cases, only getting the "build-order" dependencies is enough (e.g. to see what impact a package rebuild would have).
Add a new environment variable to disable drawing transitive dependencies.
I've marked it as Changes Requested in PAtchwork, since I have a cleaner patch to come soon, with other cleanups in the graph-depends script.
Thanks. Like I mentioned on IRC I do like the looks of the graphs, but I'm not really happy with the use of environment variables to control it, as it isn't intuitive.
I'm not a big fan of env. vars. either.
Quoted 16 lines
Quoted 2 lines
Perhaps we should simply have kconfig options for these things?
As I said on IRC, I doubt this would be usefull, For example:
- for depth: someone might want a 3-level deep graph for some package, and a 4-level deep graph for another,
- for transitive deps: someone would like the complete dependencies graphed for some package, and only the 'build-order' deps for some another.
This means the user would have to go back to the menuconfig between each graph, and this could not be made automatic (eg. called from within a script that generates the graphs for the documentation of the project.)
Quoted 7 lines
What we could do, however, would be something like:
make BR2_GRAPH_OPTS='--depth 4 --no-transitive' graph-depends(the existing
BR2_GRPAH_DEPSwould disapear).Thoughts?
An alternaitve solution could be using a config file; Python has some packages for this [1] ;-).
This way, the config file could be initialized with what is set in the menuconfig, so used as defaults.
If some options are passed on the command line, they will override these defaults.
If someone wants to do something more fancy and automatic, then one just could update the config file.
[1] https://docs.python.org/2.7/library/configparser.html
Regards,
Signature · 1 line
Samuel
PATCH 1/1 graphs: add option to remove transitive dependencies in dependency graph
Samuel, All,
On 2014-05-09 13:16 +0200, Samuel Martin spake thusly:
Quoted 1 line
Quoted 1 line
On 2014-05-09 12:00 +0200, Peter Korsgaard spake thusly:
[--SNIP--]
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Thanks. Like I mentioned on IRC I do like the looks of the graphs, but I'm not really happy with the use of environment variables to control it, as it isn't intuitive.
I'm not a big fan of env. vars. either.
[--SNIP--]
Quoted 8 lines
An alternaitve solution could be using a config file; Python has some packages for this [1] ;-).
This way, the config file could be initialized with what is set in the menuconfig, so used as defaults.
If some options are passed on the command line, they will override these defaults.
If someone wants to do something more fancy and automatic, then one just could update the config file.
Well, this does not solve the issue at hand: how do we pass options on the command line in the first place?
We added 'graph-depends' and 'PKG-graph-depends' (and 'graph-build' too) as make targets so it was easier for users to generate the graphs rather than directly call the scripts (since the scripts could even reside in another dir when building out-of-tree, and it would be more complex to reach for the scripts).
Adding a config file would not help much either: modifying the config file can't easily be automated, which is all the point in being able to pass extra args when generating the graphs.
There are people out there using a bunch of graphs to include in their projects' documentations, in an automated way.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/1 graphs: add option to remove transitive dependencies in dependency graph
Yann, all,
Quoted 20 lines · Yann E. MORIN
Samuel, All,
On 2014-05-09 13:16 +0200, Samuel Martin spake thusly:
Quoted 1 line
Quoted 1 line
On 2014-05-09 12:00 +0200, Peter Korsgaard spake thusly:
[--SNIP--]
Quoted 5 lines
Quoted 3 lines
Quoted 3 lines
Thanks. Like I mentioned on IRC I do like the looks of the graphs, but I'm not really happy with the use of environment variables to control it, as it isn't intuitive.
I'm not a big fan of env. vars. either.
[--SNIP--]
Quoted 8 lines
An alternaitve solution could be using a config file; Python has some packages for this [1] ;-).
This way, the config file could be initialized with what is set in the menuconfig, so used as defaults.
If some options are passed on the command line, they will override these defaults.
If someone wants to do something more fancy and automatic, then one just could update the config file.
Quoted 8 lines
Well, this does not solve the issue at hand: how do we pass options on the command line in the first place?
We added 'graph-depends' and 'PKG-graph-depends' (and 'graph-build' too) as make targets so it was easier for users to generate the graphs rather than directly call the scripts (since the scripts could even reside in another dir when building out-of-tree, and it would be more complex to reach for the scripts).
Arf! I forgot this.
In this case, I would do:
- to get the default graph:
make foo-graph-depends
- to get a customized graph:
make foo-graph-depends BR2_GRAPH_DEPS_OPTS="..."
And I'd expect to find some help somewhere in the doc to know what are the supported options, or how to get the graph-depends help, so via the manual or in the help message of the BR2_GRAPH_DEPS_OPTS kconfig entry.
So, in the end, the call to graph-depends script would just be:
$(TOPDIR)/support/scripts/graph-depends $(BR2_GRAPH_DEPS_OPTS)
or:
$(TOPDIR)/support/scripts/graph-depends -p $(1) $(BR2_GRAPH_DEPS_OPTS)
No black magics with the args... just keep it simple ;-) "Buildroot is a simple, efficient and easy-to-use tool" (not from me, from http://buildroot.org/ :-])
my 2 cents
Quoted 4 lines
Adding a config file would not help much either: modifying the config file can't easily be automated, which is all the point in being able to pass extra args when generating the graphs.
right.
Regards,
Signature · 1 line
Samuel
PATCH 1/1 graphs: add option to remove transitive dependencies in dependency graph
Samuel, All,
On 2014-05-09 16:09 +0200, Samuel Martin spake thusly:
Quoted 6 lines · Yann E. MORIN
Quoted 4 lines
too) as make targets so it was easier for users to generate the graphs rather than directly call the scripts (since the scripts could even reside in another dir when building out-of-tree, and it would be more complex to reach for the scripts).
Quoted 7 lines
Arf! I forgot this.
In this case, I would do:
- to get the default graph:
make foo-graph-depends
- to get a customized graph:
make foo-graph-depends BR2_GRAPH_DEPS_OPTS="..."
A single variable is not enough, since we have options:
- for support/scripts/graph-depends: maximum depth, transitive deps or not
- the 'dot' utilities: draw top-down or left-right, and so on...
So either greaph-depends has to understand a subset (or all) of dot's options (Eeek!), or we offer two variables.
Quoted 2 lines
And I'd expect to find some help somewhere in the doc to know what are the supported options,
Already in the manual, section 3.5, "Daily use" and sub-sections:
3.5.7. Graphing the dependencies between packages
http://buildroot.net/downloads/manual/manual.html#_graphing_the_dependencies_between_packages
3.5.8. Graphing the build duration
http://buildroot.net/downloads/manual/manual.html#_graphing_the_build_duration
3.5.5. Environment variables
http://buildroot.net/downloads/manual/manual.html#env-vars
Quoted 3 lines
or how to get the graph-depends help, so via
the manual or in the help message of theBR2_GRAPH_DEPS_OPTSkconfig entry.
We already document the mere existence of graph-build, graph-depends and PKG-graph-depends inthe output of make help:
<package>-graph-depends - generate graph of the dependency tree
for package
graph-build - generate graphs of the build times
graph-depends - generate graph of the dependency tree
But the variables are only documented in the manual (see above.)
Quoted 4 lines
So, in the end, the call to graph-depends script would just be:
$(TOPDIR)/support/scripts/graph-depends $(BR2_GRAPH_DEPS_OPTS)
or:
$(TOPDIR)/support/scripts/graph-depends -p $(1) $(BR2_GRAPH_DEPS_OPTS)
Not really, it currently is (roughly):
$(TOPDIR)/support/scripts/graph-depends -d $(DEPTH) \
|tee $(O)/graphs/$(@).dot \
|dot $(DOT_OPTS) -o$(O)/graphs/$(@).$(BR_GRAPH_OUT)support/scripts/graph-depends /only/ generates a dot program, not the graph itself.
Quoted 3 lines
No black magics with the args... just keep it simple ;-) "Buildroot is a simple, efficient and easy-to-use tool" (not from me, from http://buildroot.org/ :-])
Yes, but all the alternate solutions are ending being more complex that mere environment variables.
Grantes, I do not like the environment variables much, since it is easy to forget them on a subsequent make. That's why my series (I still have to post it) gets rid of as many variables as possible.
Regards,
Yann E. MORIN.
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/1 graphs: add option to remove transitive dependencies in dependency graph
Quoted 60 lines · Yann E. MORIN
Samuel, All,
On 2014-05-09 16:09 +0200, Samuel Martin spake thusly:
Quoted 6 lines · Yann E. MORIN
Quoted 4 lines
too) as make targets so it was easier for users to generate the graphs rather than directly call the scripts (since the scripts could even reside in another dir when building out-of-tree, and it would be more complex to reach for the scripts).
Quoted 7 lines
Arf! I forgot this.
In this case, I would do:
- to get the default graph:
make foo-graph-depends
- to get a customized graph:
make foo-graph-depends BR2_GRAPH_DEPS_OPTS="..."A single variable is not enough, since we have options:
- for support/scripts/graph-depends: maximum depth, transitive deps or not
- the 'dot' utilities: draw top-down or left-right, and so on...
So either greaph-depends has to understand a subset (or all) of dot's options (Eeek!), or we offer two variables.
Quoted 2 lines
And I'd expect to find some help somewhere in the doc to know what are the supported options,
Already in the manual, section 3.5, "Daily use" and sub-sections:
3.5.7. Graphing the dependencies between packages
http://buildroot.net/downloads/manual/manual.html#_graphing_the_dependencies_between_packages3.5.8. Graphing the build duration
http://buildroot.net/downloads/manual/manual.html#_graphing_the_build_duration3.5.5. Environment variables
http://buildroot.net/downloads/manual/manual.html#env-varsQuoted 3 lines
or how to get the graph-depends help, so via
the manual or in the help message of theBR2_GRAPH_DEPS_OPTSkconfig entry.We already document the mere existence of graph-build, graph-depends and PKG-graph-depends inthe output of make help:
<package>-graph-depends - generate graph of the dependency tree
for package
graph-build - generate graphs of the build times
graph-depends - generate graph of the dependency treeBut the variables are only documented in the manual (see above.)
Quoted 4 lines
So, in the end, the call to graph-depends script would just be:
$(TOPDIR)/support/scripts/graph-depends $(BR2_GRAPH_DEPS_OPTS)
or:
$(TOPDIR)/support/scripts/graph-depends -p $(1) $(BR2_GRAPH_DEPS_OPTS)
Quoted 8 lines
Not really, it currently is (roughly):
$(TOPDIR)/support/scripts/graph-depends -d $(DEPTH) \ |tee $(O)/graphs/$(@).dot \ |dot $(DOT_OPTS) -o$(O)/graphs/$(@).$(BR_GRAPH_OUT)support/scripts/graph-depends /only/ generates a dot program, not the graph itself.
/me thinking at the same time i'm writing...
It's a bit weird to me to have to pass dot options... but it's dot that generate the graph, so I see 2 options:
- in all cases, I think it's better to have a choice to select the output format;
- option 1) BR2_GRAPH_DEPS_OPTS accepts dot options after "--", so the graph-depends script should call itself the dot program;
- option 2) BR2_GRAPH_DEPS_OPTS is for graph-depends only, and we add a 3rd kconfig entry for dot option.
Wild thought: do we check for the presence of dot program?
Signature · 1 line
Samuel