Support of PRIVIDES
cover Support of PRIVIDES
Hi,
This patchset adds support of PROVIDES to Toaster backend and frontend. Please looks at https://bugzilla.yoctoproject.org/show_bug.cgi?id=6169 to see more details.
The following changes since commit f74e9383daaba1bc8eb761579f01fadc48771f4a:
toaster-manual: describe the way to start Toaster (2015-12-29 12:53:04 +0200)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib ed/toaster/provides-6169
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/provides-6169
Ed Bartosh (8):
taskdata: refactor get_providermap
cooker: add providermap to dep_tree
buildinfohelper: use providermap
toaster: add Provider model
buildinfohelper: add porvides info to the db
toaster: show list of provides for the recipe
toaster: show 'satisfied via' text for build deps
toaster: show 'satisfied via' text for reverse deps
bitbake/lib/bb/cooker.py | 7 +++++ bitbake/lib/bb/runqueue.py | 2 +- bitbake/lib/bb/taskdata.py | 17 +++++------- bitbake/lib/bb/ui/buildinfohelper.py | 31 +++++++++++++++------- .../orm/migrations/0002_auto_20151223_1528.py | 27 +++++++++++++++++++ bitbake/lib/toaster/orm/models.py | 5 ++++ .../lib/toaster/toastergui/templates/recipe.html | 25 +++++++++++++++-- 7 files changed, 92 insertions(+), 22 deletions(-) create mode 100644 bitbake/lib/toaster/orm/migrations/0002_auto_20151223_1528.py
--
Regards,
Ed
PATCH 1/8 taskdata: refactor get_providermap
Added optional parameter 'prefix' to filter out names that don't start with specified prefix. Changed existing call of get_providermap according to changed API.
Optimized the code: got rid of extra loop and temporary list variable virts.
--- bitbake/lib/bb/runqueue.py | 2 +- bitbake/lib/bb/taskdata.py | 17 +++++++---------- 2 files changed, 8 insertions(+), 11 deletions(-)
bitbake/lib/bb/runqueue.py +1 -1
@@ -808,7 +808,7 @@ class RunQueueData: invalidate_task(fn, st, True) # Create and print to the logs a virtual/xxxx -> PN (fn) table - virtmap = taskData.get_providermap() + virtmap = taskData.get_providermap(prefix="virtual/") virtpnmap = {} for v in virtmap: virtpnmap[v] = self.dataCache.pkg_fn[virtmap[v]]
bitbake/lib/bb/taskdata.py +7 -10
@@ -612,17 +612,14 @@ class TaskData: break # self.dump_data() - def get_providermap(self): - virts = [] - virtmap = {} - + def get_providermap(self, prefix=None): + provmap = {} for name in self.build_names_index: - if name.startswith("virtual/"): - virts.append(name) - for v in virts: - if self.have_build_target(v): - virtmap[v] = self.fn_index[self.get_provider(v)[0]] - return virtmap + if prefix and not name.startswith(prefix): + continue + if self.have_build_target(name): + provmap[name] = self.fn_index[self.get_provider(name)[0]] + return provmap def dump_data(self): """
Signature · 1 line
2.1.4
PATCH 2/8 cooker: add providermap to dep_tree
In reply to Ed BartoshAdded providermap information to the result of buildDependTree API.
This will be used by Toaster to map virtual dependencies to recipes.
--- bitbake/lib/bb/cooker.py | 7 +++++++ 1 file changed, 7 insertions(+)
bitbake/lib/bb/cooker.py +7 -0
@@ -717,8 +717,15 @@ class BBCooker: depend_tree["packages"] = {} depend_tree["rdepends-pkg"] = {} depend_tree["rrecs-pkg"] = {} + depend_tree['providermap'] = {} depend_tree["layer-priorities"] = self.recipecache.bbfile_config_priorities + for name, fn in taskdata.get_providermap().iteritems(): + pn = self.recipecache.pkg_fn[fn] + if name != pn: + version = "%s:%s-%s" % self.recipecache.pkg_pepvpr[fn] + depend_tree['providermap'][name] = (pn, version) + for task in xrange(len(rq.rqdata.runq_fnid)): taskname = rq.rqdata.runq_task[task] fnid = rq.rqdata.runq_fnid[task]
Signature · 1 line
2.1.4
PATCH 3/8 buildinfohelper: use providermap
In reply to Ed BartoshUsed providermap in store_dependency_information function to find virtual dependencies. This should fix annoying warnings "stpd: KeyError saving recipe dependency"
[YOCTO #6169]
--- bitbake/lib/bb/ui/buildinfohelper.py | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +16 -8
@@ -1186,6 +1186,7 @@ class BuildInfoHelper(object): assert 'layer-priorities' in event._depgraph assert 'pn' in event._depgraph assert 'tdepends' in event._depgraph + assert 'providermap' in event._depgraph errormsg = ""
@@ -1263,15 +1264,22 @@ class BuildInfoHelper(object): # buildtime recipedeps_objects = [] for recipe in event._depgraph['depends']: - try: - target = self.internal_state['recipes'][recipe] - for dep in event._depgraph['depends'][recipe]: + target = self.internal_state['recipes'][recipe] + for dep in event._depgraph['depends'][recipe]: + if dep in assume_provided: + continue + if dep in event._depgraph['providermap']: + dep = event._depgraph['providermap'][dep][0] + if dep in self.internal_state['recipes']: dependency = self.internal_state['recipes'][dep] - recipedeps_objects.append(Recipe_Dependency( recipe = target, - depends_on = dependency, dep_type = Recipe_Dependency.TYPE_DEPENDS)) - except KeyError as e: - if e not in assume_provided and not str(e).startswith("virtual/"): - errormsg += " stpd: KeyError saving recipe dependency for %s, %s \n" % (recipe, e) + else: + errormsg += " stpd: KeyError saving recipe dependency for %s, %s \n" % (recipe, dep) + continue + recipe_dep = Recipe_Dependency(recipe=target, + depends_on=dependency, + dep_type=Recipe_Dependency.TYPE_DEPENDS) + recipedeps_objects.append(recipe_dep) + Recipe_Dependency.objects.bulk_create(recipedeps_objects) # save all task information
Signature · 1 line
2.1.4
PATCH 4/8 toaster: add Provider model
In reply to Ed BartoshAdded new model Provider and a foreign key 'via' to link Recipe_Dependency to it.
[YOCTO #6169]
--- .../orm/migrations/0002_auto_20151223_1528.py | 27 ++++++++++++++++++++++ bitbake/lib/toaster/orm/models.py | 5 ++++ 2 files changed, 32 insertions(+) create mode 100644 bitbake/lib/toaster/orm/migrations/0002_auto_20151223_1528.py
bitbake/lib/toaster/orm/migrations/0002_auto_20151223_1528.py new file +27 -0
@@ -0,0 +1,27 @@ +# -*- coding: utf-8 -*- +from __future__ import unicode_literals + +from django.db import migrations, models + + +class Migration(migrations.Migration): + + dependencies = [ + ('orm', '0001_initial'), + ] + + operations = [ + migrations.CreateModel( + name='Provides', + fields=[ + ('id', models.AutoField(verbose_name='ID', serialize=False, auto_created=True, primary_key=True)), + ('name', models.CharField(max_length=100)), + ('recipe', models.ForeignKey(to='orm.Recipe')), + ], + ), + migrations.AddField( + model_name='recipe_dependency', + name='via', + field=models.ForeignKey(null=True, default=None, to='orm.Provides'), + ), + ]
bitbake/lib/toaster/orm/models.py +5 -0
@@ -733,6 +733,10 @@ class Recipe_DependencyManager(models.Manager): def get_queryset(self): return super(Recipe_DependencyManager, self).get_queryset().exclude(recipe_id = F('depends_on__id')) +class Provides(models.Model): + name = models.CharField(max_length=100) + recipe = models.ForeignKey(Recipe) + class Recipe_Dependency(models.Model): TYPE_DEPENDS = 0 TYPE_RDEPENDS = 1
@@ -743,6 +747,7 @@ class Recipe_Dependency(models.Model): ) recipe = models.ForeignKey(Recipe, related_name='r_dependencies_recipe') depends_on = models.ForeignKey(Recipe, related_name='r_dependencies_depends') + via = models.ForeignKey(Provides, null=True, default=None) dep_type = models.IntegerField(choices=DEPENDS_TYPE) objects = Recipe_DependencyManager()
Signature · 1 line
2.1.4
PATCH 5/8 buildinfohelper: add porvides info to the db
In reply to Ed BartoshAdded new entries to Provides model and link them to Recipe_Dependency using 'via' field.
This data will be used by Toaster UI to show 'Provides:' information for the recipes.
[YOCTO #6169]
--- bitbake/lib/bb/ui/buildinfohelper.py | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-)
bitbake/lib/bb/ui/buildinfohelper.py +8 -3
@@ -41,7 +41,7 @@ from orm.models import Target_Image_File, BuildArtifact from orm.models import Variable, VariableHistory from orm.models import Package, Package_File, Target_Installed_Package, Target_File from orm.models import Task_Dependency, Package_Dependency -from orm.models import Recipe_Dependency +from orm.models import Recipe_Dependency, Provides from orm.models import Project from bldcontrol.models import BuildEnvironment, BuildRequest
@@ -1268,15 +1268,20 @@ class BuildInfoHelper(object): for dep in event._depgraph['depends'][recipe]: if dep in assume_provided: continue + via = None if dep in event._depgraph['providermap']: - dep = event._depgraph['providermap'][dep][0] - if dep in self.internal_state['recipes']: + deprecipe = event._depgraph['providermap'][dep][0] + dependency = self.internal_state['recipes'][deprecipe] + via = Provides.objects.get_or_create(name=dep, + recipe=dependency)[0] + elif dep in self.internal_state['recipes']: dependency = self.internal_state['recipes'][dep] else: errormsg += " stpd: KeyError saving recipe dependency for %s, %s \n" % (recipe, dep) continue recipe_dep = Recipe_Dependency(recipe=target, depends_on=dependency, + via=via, dep_type=Recipe_Dependency.TYPE_DEPENDS) recipedeps_objects.append(recipe_dep)
Signature · 1 line
2.1.4
PATCH 6/8 toaster: show list of provides for the recipe
In reply to Ed BartoshShowed list of names that recipe provides.
--- bitbake/lib/toaster/toastergui/templates/recipe.html | 7 +++++++ 1 file changed, 7 insertions(+)
bitbake/lib/toaster/toastergui/templates/recipe.html +7 -0
@@ -71,6 +71,13 @@ Layer commit </dt> <dd class="iscommit">{{layer_version.commit}}</dd> + {% if object.provides_set.all %} + <dt> + <i class="icon-question-sign get-help" title="List of names that this recipe privides"></i> + PROVIDES + </dt> + <dd><code>{% for provider in object.provides_set.all %}{{ provider.name }} {% endfor %}</code></dd> + {% endif %} </dl> <h2 class="details">Tasks</h2>
Signature · 1 line
2.1.4
PATCH 7/8 toaster: show 'satisfied via' text for build deps
In reply to Ed BartoshShowed '<dependency> satisfied via <provider>' text and help tooltip for the build dependencies provided through 'PROVIDES' in the 'Build dependencies' tab.
[YOCTO #6169]
--- bitbake/lib/toaster/toastergui/templates/recipe.html | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
bitbake/lib/toaster/toastergui/templates/recipe.html +8 -1
@@ -169,7 +169,14 @@ {% for rr in object.r_dependencies_recipe.all|dictsort:"depends_on.name" %} <tr> - <td><a href="{% url "recipe" build.pk rr.depends_on.pk %}">{{rr.depends_on.name}}</a></td> + <td><a href="{% url "recipe" build.pk rr.depends_on.pk %}">{{rr.depends_on.name}} + {% if rr.via %} + <span class="muted"> satisfied via {{rr.via.name}}</span> + <i class="icon-question-sign get-help" + title="This dependency is satisfied by the PROVIDES value + {{rr.via.name}} in the {{rr.depends_on.name}} recipe"></i> + {% endif %} + </a></td> <td><a href="{% url "recipe" build.pk rr.depends_on.pk %}">{{rr.depends_on.version}}</a></td> </tr> {% endfor %}
Signature · 1 line
2.1.4
PATCH 8/8 toaster: show 'satisfied via' text for reverse deps
In reply to Ed BartoshShowed '<dependency> satisfied via <provider>' text and help tooltip for the revrse build dependencies provided through 'PROVIDES' in the 'Reverse build dependencies' tab.
[YOCTO #6169]
--- bitbake/lib/toaster/toastergui/templates/recipe.html | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
bitbake/lib/toaster/toastergui/templates/recipe.html +8 -1
@@ -208,7 +208,14 @@ {% for rr in object.r_dependencies_depends.all|dictsort:"recipe.name" %} <tr> - <td><a href="{% url "recipe" build.pk rr.recipe.pk %}">{{rr.recipe.name}}</a></td> + <td><a href="{% url "recipe" build.pk rr.recipe.pk %}">{{rr.recipe.name}} + {% if rr.via %} + <span class="muted"> satisfied via {{rr.via.name}}</span> + <i class="icon-question-sign get-help" + title="This dependency is satisfied by the PROVIDES value + {{rr.via.name}} in the {{rr.depends_on.name}} recipe"></i> + {% endif %} + </a></td> <td><a href="{% url "recipe" build.pk rr.recipe.pk %}">{{rr.recipe.version}}</a></td> </tr> {% endfor %}
Signature · 1 line
2.1.4
Quoted 12 lines · On 29/12/2015 11:56, "[email protected] on behalf of Ed…
Hi,
This patchset adds support of PROVIDES to Toaster backend and frontend. Please looks at https://bugzilla.yoctoproject.org/show_bug.cgi?id=6169 to see more details.
The following changes since commit
f74e9383daaba1bc8eb761579f01fadc48771f4a:toaster-manual: describe the way to start Toaster (2015-12-29 12:53:04
+0200)
Quoted 6 lines
are available in the git repository at:
git://git.yoctoproject.org/poky-contribed/toaster/provides-6169http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/provid
es-6169
This looks good to me. Just a couple of small presentation issues:
* In the 'recipe details' tab, the help text for the 'PROVIDES' entry should say
A list of aliases by which a particular recipe can be known. The additional aliases are synonyms for the recipe and can be useful satisfying dependencies of other recipes during the build
Right now it says
List of names that this recipe privides
* In the dependencies tabs, only the recipe name should be inside the anchor tag. Right now, the 'satisfied via ...' it's also inside the anchor tag, and that creates an uncomfortable effect when you hover in order to click the recipe name.
* Also in the dependencies tabs, the question mark icon should only show in the table when you hover over the cell (we do this across all tables and keeps the tables a bit cleaner). You just need to apply the hover-help class to the <i> tag
That's all. Thanks!
Belén
Quoted 33 lines
Ed Bartosh (8):
taskdata: refactorget_providermap
cooker: add providermap todep_tree
buildinfohelper: use providermap
toaster: add Provider model
buildinfohelper: add porvides info to the db
toaster: show list of provides for the recipe
toaster: show 'satisfied via' text for build deps
toaster: show 'satisfied via' text for reverse depsbitbake/lib/bb/cooker.py | 7 +++++ bitbake/lib/bb/runqueue.py | 2 +- bitbake/lib/bb/taskdata.py | 17 +++++------- bitbake/lib/bb/ui/buildinfohelper.py | 31+++++++++++++++-------
.../orm/migrations/0002_auto_20151223_1528.py| 27
+++++++++++++++++++bitbake/lib/toaster/orm/models.py| 5 ++++
.../lib/toaster/toastergui/templates/recipe.html | 25 +++++++++++++++--7 files changed, 92 insertions(+), 22 deletions(-)create mode 100644
bitbake/lib/toaster/orm/migrations/0002_auto_20151223_1528.py--
Regards,
EdSignature · 4 lines
_______________________________________________ toaster mailing list [email protected] https://lists.yoctoproject.org/listinfo/toaster
Hi Belen,
Thank you for review!
I've fixed the issues you've found and updated my branch ed/toaster/provides-6169
Regards,
Ed
Quoted 87 lines · Belen
Quoted 12 lines · On 29/12/2015 11:56, "[email protected] on behalf of Ed…
Hi,
This patchset adds support of PROVIDES to Toaster backend and frontend. Please looks at https://bugzilla.yoctoproject.org/show_bug.cgi?id=6169 to see more details.
The following changes since commit
f74e9383daaba1bc8eb761579f01fadc48771f4a:toaster-manual: describe the way to start Toaster (2015-12-29 12:53:04
+0200)Quoted 6 lines
are available in the git repository at:
git://git.yoctoproject.org/poky-contribed/toaster/provides-6169http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/provid
es-6169This looks good to me. Just a couple of small presentation issues:
* In the 'recipe details' tab, the help text for the 'PROVIDES' entry should say
A list of aliases by which a particular recipe can be known. The additional aliases are synonyms for the recipe and can be useful satisfying dependencies of other recipes during the build
Right now it says
List of names that this recipe privides
* In the dependencies tabs, only the recipe name should be inside the anchor tag. Right now, the 'satisfied via ...' it's also inside the anchor tag, and that creates an uncomfortable effect when you hover in order to click the recipe name.
* Also in the dependencies tabs, the question mark icon should only show in the table when you hover over the cell (we do this across all tables and keeps the tables a bit cleaner). You just need to apply the hover-help class to the <i> tag
That's all. Thanks!
Belén
Quoted 33 lines
Ed Bartosh (8):
taskdata: refactorget_providermap
cooker: add providermap todep_tree
buildinfohelper: use providermap
toaster: add Provider model
buildinfohelper: add porvides info to the db
toaster: show list of provides for the recipe
toaster: show 'satisfied via' text for build deps
toaster: show 'satisfied via' text for reverse depsbitbake/lib/bb/cooker.py | 7 +++++ bitbake/lib/bb/runqueue.py | 2 +- bitbake/lib/bb/taskdata.py | 17 +++++------- bitbake/lib/bb/ui/buildinfohelper.py | 31+++++++++++++++-------
.../orm/migrations/0002_auto_20151223_1528.py| 27
+++++++++++++++++++bitbake/lib/toaster/orm/models.py| 5 ++++
.../lib/toaster/toastergui/templates/recipe.html | 25 +++++++++++++++--7 files changed, 92 insertions(+), 22 deletions(-)create mode 100644
bitbake/lib/toaster/orm/migrations/0002_auto_20151223_1528.py--
Regards,
EdSignature · 4 lines
_______________________________________________ toaster mailing list [email protected] https://lists.yoctoproject.org/listinfo/toaster
Signature · 3 lines
-- Regards, Ed
Quoted 6 lines · Ed Bartosh
Hi Belen,
Thank you for review!
I've fixed the issues you've found and updated my branch ed/toaster/provides-6169
Thanks for the v2. Looks good to me now.
Cheers
Belén
Quoted 104 lines
Regards,
EdQuoted 94 lines · Belen
Quoted 4 lines · On 29/12/2015 11:56, "[email protected] on behalf of Ed…
Hi,
This patchset adds support of PROVIDES to Toaster backend and frontend. Please looks at https://bugzilla.yoctoproject.org/show_bug.cgi?id=6169
to
Quoted 6 lines
see more details.
The following changes since commit
f74e9383daaba1bc8eb761579f01fadc48771f4a:toaster-manual: describe the way to start Toaster (2015-12-29
12:53:04
Quoted 10 lines
+0200)
are available in the git repository at:
git://git.yoctoproject.org/poky-contribed/toaster/provides-6169http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/prov
id
es-6169This looks good to me. Just a couple of small presentation issues:
* In the 'recipe details' tab, the help text for the 'PROVIDES' entry should say
A list of aliases by which a particular recipe can be known. The additional aliases are synonyms for the recipe and can be useful satisfying dependencies of other recipes during the build
Right now it says
List of names that this recipe privides
* In the dependencies tabs, only the recipe name should be inside the anchor tag. Right now, the 'satisfied via ...' it's also inside the anchor
tag, and that creates an uncomfortable effect when you hover in order to click the recipe name.* Also in the dependencies tabs, the question mark icon should only show in the table when you hover over the cell (we do this across all tables and keeps the tables a bit cleaner). You just need to apply the hover-help
class to the <i> tagThat's all. Thanks!
Belén
Quoted 15 lines
Ed Bartosh (8):
taskdata: refactorget_providermap
cooker: add providermap todep_tree
buildinfohelper: use providermap
toaster: add Provider model
buildinfohelper: add porvides info to the db
toaster: show list of provides for the recipe
toaster: show 'satisfied via' text for build deps
toaster: show 'satisfied via' text for reverse depsbitbake/lib/bb/cooker.py | 7 +++++ bitbake/lib/bb/runqueue.py | 2 +- bitbake/lib/bb/taskdata.py | 17 +++++------- bitbake/lib/bb/ui/buildinfohelper.py | 31Quoted 5 lines
+++++++++++++++-------
.../orm/migrations/0002_auto_20151223_1528.py| 27
+++++++++++++++++++bitbake/lib/toaster/orm/models.py| 5 ++++
.../lib/toaster/toastergui/templates/recipe.html | 25+++++++++++++++--
Quoted 13 lines
7 files changed, 92 insertions(+), 22 deletions(-)create mode 100644
bitbake/lib/toaster/orm/migrations/0002_auto_20151223_1528.py--
Regards,
EdSignature · 4 lines
_______________________________________________ toaster mailing list [email protected] https://lists.yoctoproject.org/listinfo/toasterSignature · 3 lines
-- Regards, Ed
Quoted 4 lines · Ed Bartosh
This patchset adds support of PROVIDES to Toaster backend and frontend. Please looks at https://bugzilla.yoctoproject.org/show_bug.cgi?id=6169 to see more details.
Submitted to bitbake-devel and merged into toaster-next.
Elliot
Quoted 46 lines
The following changes since commit
f74e9383daaba1bc8eb761579f01fadc48771f4a:toaster-manual: describe the way to start Toaster (2015-12-29 12:53:04
+0200)are available in the git repository at:
git://git.yoctoproject.org/poky-contribed/toaster/provides-6169http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/provides-6169
Ed Bartosh (8):
taskdata: refactorget_providermap
cooker: add providermap todep_tree
buildinfohelper: use providermap
toaster: add Provider model
buildinfohelper: add porvides info to the db
toaster: show list of provides for the recipe
toaster: show 'satisfied via' text for build deps
toaster: show 'satisfied via' text for reverse depsbitbake/lib/bb/cooker.py | 7 +++++ bitbake/lib/bb/runqueue.py | 2 +- bitbake/lib/bb/taskdata.py | 17 +++++------- bitbake/lib/bb/ui/buildinfohelper.py | 31 +++++++++++++++------- .../orm/migrations/0002_auto_20151223_1528.py | 27 +++++++++++++++++++ bitbake/lib/toaster/orm/models.py | 5 ++++ .../lib/toaster/toastergui/templates/recipe.html | 25 +++++++++++++++-- 7 files changed, 92 insertions(+), 22 deletions(-) create mode 100644
bitbake/lib/toaster/orm/migrations/0002_auto_20151223_1528.py--
Regards,
Ed--
_______________________________________________
toaster mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/toaster
Signature · 3 lines
Elliot Smith Software Engineer Intel Open Source Technology Centre