Skip to contentSkip to conversation

toaster patchset

PATCH New Alex DAMIAN 1 participant 19 messages Mar 24, 2014

lore
Share & tools
lore
#0 lore raw

cover toaster patchset

From: Alexandru DAMIAN <[email protected]>

This is a pull-request for toaster. It includes fixes for data collection and for the UI. The patches have been reviewed on the toaster mailing list.

Can you please merge this ?

Thank you,
Alex

The following changes since commit f34575809677dc52e1071a3ae3daebe92819cec0:

hob: output filenames based on initial recipe name (2014-03-24 10:13:58 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib adamian/20140324-submission-bb
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=adamian/20140324-submission-bb

Alexandru DAMIAN (5):
toaster: Revert "added file types to the Outputs column in the build
page"
toaster: select recipe based on PN name
toasterui: save missed sstate tasks
toaster: clean exit on bb server shutdown
toaster: replace package dependency tag w/ view queries

Amit Kumar Chaudhary (1):
toaster: combine ready functions

Belen Barros Pena (5):
toaster: Change objectname 'packages' to 'packages built'
toaster: Fix alignment issue in searchform
toaster: Add help text to filters
toaster: Update local configuration counter
toaster: Change "0 found" to "No found"

Cristiana Voicu (1):
bitbake toaster: check the file_name with the content of the
IMAGE_FSTYPES variable

Dave Lerner (1):
toaster: fix package size 1st sort order

David Reyna (3):
toaster: warn new filter replaces existing filter
toaster: reset default filter for configvar page on new searches
toaster: add search by section to all recipe page

Irina Patru (1):
toaster: Remove circular dependecies from packages/recipes

Marius Avram (1):
toaster: apply filter only on selected attributes

 bin/toaster                                        | 28 +++++++++++-
 lib/bb/ui/buildinfohelper.py                       | 50 ++++++++++++++++------
 lib/bb/ui/toasterui.py                             |  9 +++-
 lib/toaster/orm/models.py                          | 16 ++++++-
 lib/toaster/toastergui/static/css/default.css      |  1 +
 lib/toaster/toastergui/static/js/main.js           |  4 +-
 .../toastergui/templates/basetable_top.html        | 10 ++---
 lib/toaster/toastergui/templates/bpackage.html     |  6 +--
 lib/toaster/toastergui/templates/build.html        |  6 +--
 lib/toaster/toastergui/templates/configvars.html   |  6 +--
 .../toastergui/templates/filtersnippet.html        |  6 +++
 lib/toaster/toastergui/templates/recipes.html      |  6 +--
 lib/toaster/toastergui/templates/target.html       |  4 +-
 lib/toaster/toastergui/templates/tasks.html        |  6 +--
 lib/toaster/toastergui/templatetags/projecttags.py | 41 ++----------------
 lib/toaster/toastergui/views.py                    | 49 ++++++++++++---------
 16 files changed, 144 insertions(+), 104 deletions(-)
Signature · 1 line
1.8.3.2
#1 lore raw

PATCH 1/18 toaster: Revert "added file types to the Outputs column in the build page"

From: Alexandru DAMIAN <[email protected]>

This reverts commit d40ac966b22e1fa1956d8f2fe37fd55fa670e88f.

Sloppy review on my part let the original patch in, when it should've been rejected because the filter tags should have no knowledge of the object system.

---
 lib/toaster/toastergui/templates/build.html        |  6 +----
 lib/toaster/toastergui/templatetags/projecttags.py | 27 ----------------------
 2 files changed, 1 insertion(+), 32 deletions(-)
lib/toaster/toastergui/templates/build.html +1 -5
index 1c94e7c..8f557e9 100644
@@ -92,11 +92,7 @@
             <td class="warnings_no">{% if  build.warnings_no %}<a class="warnings_no warning" href="{% url "builddashboard" build.id %}#warnings">{{build.warnings_no}} warning{{build.warnings_no|pluralize}}</a>{%endif%}</td>
             <td class="time"><a href="{% url "buildtime" build.id %}">{{build.timespent|sectohms}}</a></td>
             <td class="log">{{build.cooker_log_path}}</td>
-            <td class="output">
-              {% if build.outcome == 0 %}
-              {{build|get_image_extensions}}
-              {% endif %}
-            </td>
+            <td class="output">{% if build.outcome == 0 %}{% for t in build.target_set.all %}{% if t.is_image %}<a href="{%url "builddashboard" build.id%}#images">TODO: compute image output fstypes</a>{% endif %}{% endfor %}{% endif %}</td>
         </tr>
 
         {% endfor %}
lib/toaster/toastergui/templatetags/projecttags.py +0 -27
index e08258b..e7b23cb 100644
@@ -24,8 +24,6 @@ import re
 from django import template
 from django.utils import timezone
 from django.template.defaultfilters import filesizeformat
-from orm.models import Target_Installed_Package, Target_Image_File
-from orm.models import Build, Target, Task, Layer, Layer_Version
 
 register = template.Library()
 
@@ -191,31 +189,6 @@ def string_slice(strvar,slicevar):
         return strvar[int(first):int(last)]
 
 @register.filter
-def get_image_extensions( build ):
-    """
-    This is a simple filter that returns a list (string)
-    of extensions of the build-targets-image files. Note
-    that each build can have multiple targets and each
-    target can yield more than one image file
-    """
-    targets = Target.objects.filter( build_id = build.id );
-    comma = "";
-    extensions = "";
-    for t in targets:
-        if ( not t.is_image ):
-            continue;
-        tif = Target_Image_File.objects.filter( target_id = t.id );
-        for i in tif:
-            try:
-                ndx = i.file_name.index( "." );
-            except ValueError:
-                ndx = 0;
-            s = i.file_name[ ndx + 1 : ];
-            extensions += comma + s;
-            comma = ", ";
-    return( extensions );
-
[email protected]
 def filtered_installedsize(size, installed_size):
     """If package.installed_size not null and not empty return it,
        else return package.size
Signature · 1 line
1.8.3.2
#2 lore raw

PATCH 2/18 toaster: warn new filter replaces existing filter

From: David Reyna <[email protected]>

If the user pop-ups a filter dialog when another filter is already active, warn the user that the new filter would replace the previous filter.

[YOCTO #5960]

---
 lib/toaster/toastergui/static/css/default.css       | 1 +
 lib/toaster/toastergui/templates/filtersnippet.html | 5 +++++
 lib/toaster/toastergui/templatetags/projecttags.py  | 6 ++++++
 3 files changed, 12 insertions(+)
lib/toaster/toastergui/static/css/default.css +1 -0
index 53a3fee..d960f4b 100644
@@ -105,6 +105,7 @@ select { width: auto; }
 .no-results { margin: 10px 0; }
 .task-name { margin-left: 7px; }
 .icon-hand-right {color: #ccccc; }
+.help-inline { margin: 5px; }
 
 
 
lib/toaster/toastergui/templates/filtersnippet.html +5 -0
index e0e1181..a72eeaa 100644
@@ -28,6 +28,11 @@
         </div>
         <div class="modal-footer">
             <button type="submit" class="btn btn-primary">Apply</button>
+            {% if request.GET.filter %}
+                {% if request.GET.filter|string_remove_regex:':.*' != f.options.0.1|string_remove_regex:':.*' %}
+                <span class="help-inline pull-left">You can only apply one filter to the table. This filter will override the current filter.</span>
+                {% endif %}
+            {% endif %}
         </div>
 </form>
 
lib/toaster/toastergui/templatetags/projecttags.py +6 -0
index e7b23cb..39e92ce 100644
@@ -189,6 +189,12 @@ def string_slice(strvar,slicevar):
         return strvar[int(first):int(last)]
 
 @register.filter
+def string_remove_regex(value,ex):
+    """ remove sub-string of string that matches regex
+    """
+    return re.sub(ex, '', value)
+
[email protected]
 def filtered_installedsize(size, installed_size):
     """If package.installed_size not null and not empty return it,
        else return package.size
Signature · 1 line
1.8.3.2
#3 lore raw

PATCH 3/18 toaster: reset default filter for configvar page on new searches

In reply to Alex DAMIAN

From: David Reyna <[email protected]>

The configvar page has a default filter by design. However, new search terms should override and clear that filter, for consistency across the interface.

[YOCTO #5961]

---
 lib/toaster/toastergui/views.py | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
lib/toaster/toastergui/views.py +4 -1
index 97514cc..eaa6cf8 100644
@@ -1010,10 +1010,13 @@ def configvars(request, build_id):
     template = 'configvars.html'
     mandatory_parameters = { 'count': 100,  'page' : 1, 'orderby':'variable_name:+', 'filter':'description__regex:.+'};
     retval = _verify_parameters( request.GET, mandatory_parameters )
+    (filter_string, search_term, ordering_string) = _search_tuple(request, Variable)
     if retval:
+        # if new search, clear the default filter
+        if search_term and len(search_term):
+            mandatory_parameters['filter']=''
         return _redirect_parameters( 'configvars', request.GET, mandatory_parameters, build_id = build_id)
 
-    (filter_string, search_term, ordering_string) = _search_tuple(request, Variable)
     queryset = Variable.objects.filter(build=build_id).exclude(variable_name__istartswith='B_').exclude(variable_name__istartswith='do_')
     queryset_with_search =  _get_queryset(Variable, queryset, None, search_term, ordering_string).distinct().exclude(variable_value='',vhistory__file_name__isnull=True)
     queryset = _get_queryset(Variable, queryset, filter_string, search_term, ordering_string)
Signature · 1 line
1.8.3.2
#4 lore raw

PATCH 4/18 toaster: Change objectname 'packages' to 'packages built'

In reply to Alex DAMIAN

From: Belen Barros Pena <[email protected]>

This will set the placeholder attribute of the search input field in bpackage.html to the required "Search packages built".

[YOCTO #5977]

---
 lib/toaster/toastergui/views.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/toaster/toastergui/views.py +1 -1
index eaa6cf8..a4bd9d6 100644
@@ -1111,7 +1111,7 @@ def bpackage(request, build_id):
     packages = _build_page_range(Paginator(queryset, request.GET.get('count', 100)),request.GET.get('page', 1))
 
     context = {
-        'objectname': 'packages',
+        'objectname': 'packages built',
         'build': Build.objects.filter(pk=build_id)[0],
         'objects' : packages,
         'tablecols':[
Signature · 1 line
1.8.3.2
#5 lore raw

PATCH 5/18 toaster: fix package size 1st sort order

In reply to Alex DAMIAN

From: Dave Lerner <[email protected]>

[YOCTO #5914]

On the pages with tables listing packages with a size column, change the first click sort order on the size column so that packages are sorted in decreasing order instead of increasing order. It is more likely that the user sorts this page data by size because the user is interested in those packages consuming the most disk space.

---
 lib/toaster/toastergui/views.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
lib/toaster/toastergui/views.py +2 -2
index a4bd9d6..3f6b93e 100644
@@ -448,7 +448,7 @@ def target(request, build_id, target_id):
                 {
                     'name':'Size',
                     'qhelp':'The size of the package',
-                    'orderfield': _get_toggle_order(request, "size"),
+                    'orderfield': _get_toggle_order(request, "size", True),
                     'ordericon':_get_toggle_order_icon(request, "size"),
                     'clclass': 'package_size',
                     'hidden' : 0,
@@ -1128,7 +1128,7 @@ def bpackage(request, build_id):
             {
                 'name':'Size',
                 'qhelp':'The size of the package',
-                'orderfield': _get_toggle_order(request, "size"),
+                'orderfield': _get_toggle_order(request, "size", True),
                 'ordericon':_get_toggle_order_icon(request, "size"),
                 'clclass': 'size', 'hidden': 0,
             },
Signature · 1 line
1.8.3.2
#6 lore raw

PATCH 6/18 toaster: Fix alignment issue in searchform

In reply to Alex DAMIAN

From: Belen Barros Pena <[email protected]>

The search form in basetable_top.html had a small alignment issue between the search text field and the search button. This change aligns the elements correctly.

Whitespace fix by Alex Damian.

---
 lib/toaster/toastergui/templates/basetable_top.html | 8 +++-----
 1 file changed, 3 insertions(+), 5 deletions(-)
lib/toaster/toastergui/templates/basetable_top.html +3 -5
index 9df3647..58b55b6 100644
@@ -30,12 +30,10 @@
 <div class="navbar">
     <div class="navbar-inner">
         <form class="navbar-search input-append pull-left" id="searchform">
-	<div class="input-append" style="padding-right:1em">
             <input class="input-xxlarge" id="search" name="search" type="text" placeholder="Search {%if object_search_display %}{{object_search_display}}{%else%}{{objectname}}{%endif%}" value="{{request.GET.search}}"/>{% if request.GET.search %}<a href="javascript:$('#search').val('');searchform.submit()" class="add-on"><i class="icon-remove"></i></a>{%endif%}
-	</div>
-	    <input type="hidden" name="orderby" value="{{request.GET.orderby}}">
-	    <input type="hidden" name="page" value="1">
-            <input class="btn" type="submit" value="Search"/>
+            <input type="hidden" name="orderby" value="{{request.GET.orderby}}">
+            <input type="hidden" name="page" value="1">
+            <button class="btn" type="submit" value="Search">Search</button>
         </form>
         <div class="pull-right">
 {% if tablecols %}
Signature · 1 line
1.8.3.2
#7 lore raw

PATCH 7/18 toaster: Add help text to filters

In reply to Alex DAMIAN

From: Belen Barros Pena <[email protected]>

The 'Set in file' and 'Description' filters in the configuration page, and the 'Outcome' and 'Cache attempt' filters in the tasks page should include a help tooltip with some explanation about what they do. This change adds the tooltips.

---
 .../toastergui/templates/filtersnippet.html        |  1 +
 lib/toaster/toastergui/views.py                    | 32 +++++++++++-----------
 2 files changed, 17 insertions(+), 16 deletions(-)
lib/toaster/toastergui/templates/filtersnippet.html +1 -0
index a72eeaa..90ffd3d 100644
@@ -23,6 +23,7 @@
                     <label class="radio muted">
                         <input type="radio" name="filter" disabled {%if request.GET.filter == option.1 %}checked{%endif%}  value="{{option.1}}"> {{option.0}} ({{option.2}})
                 {% endif %}
+                {% if option.3 %}<i class="icon-question-sign get-help" data-placement="right" title="{{option.3}}"></i>{% endif %}
                 </label>
             {% endfor %}
         </div>
lib/toaster/toastergui/views.py +16 -16
index 3f6b93e..2156d68 100644
@@ -782,12 +782,12 @@ def tasks_common(request, build_id, variant):
                    'class' : 'outcome',
                    'label': 'Show:',
                    'options' : [
-                               ('Succeeded Tasks', 'outcome:%d'%Task.OUTCOME_SUCCESS, queryset_with_search.filter(outcome=Task.OUTCOME_SUCCESS).count() ),
-                               ('Failed Tasks', 'outcome:%d'%Task.OUTCOME_FAILED, queryset_with_search.filter(outcome=Task.OUTCOME_FAILED).count()),
-                               ('Cached Tasks', 'outcome:%d'%Task.OUTCOME_CACHED, queryset_with_search.filter(outcome=Task.OUTCOME_CACHED).count()),
-                               ('Prebuilt Tasks', 'outcome:%d'%Task.OUTCOME_PREBUILT, queryset_with_search.filter(outcome=Task.OUTCOME_PREBUILT).count()),
-                               ('Covered Tasks', 'outcome:%d'%Task.OUTCOME_COVERED, queryset_with_search.filter(outcome=Task.OUTCOME_COVERED).count()),
-                               ('Empty Tasks', 'outcome:%d'%Task.OUTCOME_EMPTY, queryset_with_search.filter(outcome=Task.OUTCOME_NA).count()),
+                               ('Succeeded Tasks', 'outcome:%d'%Task.OUTCOME_SUCCESS, queryset_with_search.filter(outcome=Task.OUTCOME_SUCCESS).count(), 'Succeeded tasks are those that ran and completed during the build' ),
+                               ('Failed Tasks', 'outcome:%d'%Task.OUTCOME_FAILED, queryset_with_search.filter(outcome=Task.OUTCOME_FAILED).count(), 'Failed tasks are those that ran but did not complete during the build'),
+                               ('Cached Tasks', 'outcome:%d'%Task.OUTCOME_CACHED, queryset_with_search.filter(outcome=Task.OUTCOME_CACHED).count(), 'Cached tasks restored output from the <code>sstate-cache</code> directory or mirrors'),
+                               ('Prebuilt Tasks', 'outcome:%d'%Task.OUTCOME_PREBUILT, queryset_with_search.filter(outcome=Task.OUTCOME_PREBUILT).count(),'Prebuilt tasks didn\'t need to run because their output was reused from a previous build'),
+                               ('Covered Tasks', 'outcome:%d'%Task.OUTCOME_COVERED, queryset_with_search.filter(outcome=Task.OUTCOME_COVERED).count(), 'Covered tasks didn\'t need to run because their output is provided by another task in this build'),
+                               ('Empty Tasks', 'outcome:%d'%Task.OUTCOME_EMPTY, queryset_with_search.filter(outcome=Task.OUTCOME_NA).count(), 'Empty tasks have no executable content'),
                                ]
                    }
 
@@ -809,10 +809,10 @@ def tasks_common(request, build_id, variant):
                    'class' : 'cache_attempt',
                    'label': 'Show:',
                    'options' : [
-                               ('Tasks with cache attempts', 'sstate_result:%d'%Task.SSTATE_NA, queryset_with_search.filter(sstate_result=Task.SSTATE_NA).count()),
-                               ("Tasks with 'File not in cache' attempts", 'sstate_result:%d'%Task.SSTATE_MISS,  queryset_with_search.filter(sstate_result=Task.SSTATE_MISS).count()),
-                               ("Tasks with 'Failed' cache attempts", 'sstate_result:%d'%Task.SSTATE_FAILED,  queryset_with_search.filter(sstate_result=Task.SSTATE_FAILED).count()),
-                               ("Tasks with 'Succeeded' cache attempts", 'sstate_result:%d'%Task.SSTATE_RESTORED,  queryset_with_search.filter(sstate_result=Task.SSTATE_RESTORED).count()),
+                               ('Tasks with cache attempts', 'sstate_result:%d'%Task.SSTATE_NA, queryset_with_search.filter(sstate_result=Task.SSTATE_NA).count(), 'Show all tasks that tried to restore ouput from the <code>sstate-cache</code> directory or mirrors'),
+                               ("Tasks with 'File not in cache' attempts", 'sstate_result:%d'%Task.SSTATE_MISS,  queryset_with_search.filter(sstate_result=Task.SSTATE_MISS).count(), 'Show tasks that tried to restore output, but did not find it in the <code>sstate-cache</code> directory or mirrors'),
+                               ("Tasks with 'Failed' cache attempts", 'sstate_result:%d'%Task.SSTATE_FAILED,  queryset_with_search.filter(sstate_result=Task.SSTATE_FAILED).count(), 'Show tasks that found the required output in the <code>sstate-cache</code> directory or mirrors, but could not restore it'),
+                               ("Tasks with 'Succeeded' cache attempts", 'sstate_result:%d'%Task.SSTATE_RESTORED,  queryset_with_search.filter(sstate_result=Task.SSTATE_RESTORED).count(), 'Show tasks that successfully restored the required output from the <code>sstate-cache</code> directory or mirrors'),
                                ]
                    }
 
@@ -1072,11 +1072,11 @@ def configvars(request, build_id):
                     'class' : 'vhistory__file_name',
                     'label': 'Show:',
                     'options' : [
-                               ('Local configuration variables', 'vhistory__file_name__regex:conf/(local|bblayers).conf', queryset_with_search.filter(vhistory__file_name__contains='conf/local.conf').count()),
-                               ('Machine configuration variables', 'vhistory__file_name__contains:conf/machine/',queryset_with_search.filter(vhistory__file_name__contains='conf/machine').count()),
-                               ('Distro configuration variables', 'vhistory__file_name__contains:conf/distro/',queryset_with_search.filter(vhistory__file_name__contains='conf/distro').count()),
-                               ('Layer configuration variables', 'vhistory__file_name__contains:conf/layer.conf',queryset_with_search.filter(vhistory__file_name__contains='conf/layer.conf').count()),
-                               ('bitbake.conf variables', 'vhistory__file_name__contains:/bitbake.conf',queryset_with_search.filter(vhistory__file_name__contains='/bitbake.conf').count()),
+                               ('Local configuration variables', 'vhistory__file_name__regex:conf/(local|bblayers).conf', queryset_with_search.filter(vhistory__file_name__contains='conf/local.conf').count(), 'Select this filter to see variables set by the <code>local.conf</code> and <code>bblayers.conf</code> configuration files inside the <code>/build/conf/</code> directory'),
+                               ('Machine configuration variables', 'vhistory__file_name__contains:conf/machine/',queryset_with_search.filter(vhistory__file_name__contains='conf/machine').count(), 'Select this filter to see variables set by the configuration file(s) inside your layers <code>/conf/machine/</code> directory'),
+                               ('Distro configuration variables', 'vhistory__file_name__contains:conf/distro/',queryset_with_search.filter(vhistory__file_name__contains='conf/distro').count(), 'Select this filter to see variables set by the configuration file(s) inside your layers <code>/conf/distro/</code> directory'),
+                               ('Layer configuration variables', 'vhistory__file_name__contains:conf/layer.conf',queryset_with_search.filter(vhistory__file_name__contains='conf/layer.conf').count(), 'Select this filter to see variables set by the <code>layer.conf</code> configuration file inside your layers'),
+                               ('bitbake.conf variables', 'vhistory__file_name__contains:/bitbake.conf',queryset_with_search.filter(vhistory__file_name__contains='/bitbake.conf').count(), 'Select this filter to see variables set by the <code>bitbake.conf</code> configuration file'),
                                ]
                              },
                 },
@@ -1088,7 +1088,7 @@ def configvars(request, build_id):
                     'class' : 'description',
                     'label': 'Show:',
                     'options' : [
-                               ('Variables with description', 'description__regex:.+', queryset_with_search.filter(description__regex='.+').count()),
+                               ('Variables with description', 'description__regex:.+', queryset_with_search.filter(description__regex='.+').count(), 'We provide descriptions for the most common BiBake variables, so this filter effectively shows you a curated list of variables put together by the Yocto Project maintainers. The curated list lives in <code>/meta/conf/documentation.conf</code> inside your <code>poky</code> directory'),
                                ]
                             },
                 },
Signature · 1 line
1.8.3.2
#8 lore raw

PATCH 8/18 toaster: Update local configuration counter

In reply to Alex DAMIAN

From: Belen Barros Pena <[email protected]>

The counter for the local configuration filter in the variables table was counting only variables set by local.conf. Updating to add in variables set by bblayers.conf

---
 lib/toaster/toastergui/views.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/toaster/toastergui/views.py +1 -1
index 2156d68..732be8f 100644
@@ -1072,7 +1072,7 @@ def configvars(request, build_id):
                     'class' : 'vhistory__file_name',
                     'label': 'Show:',
                     'options' : [
-                               ('Local configuration variables', 'vhistory__file_name__regex:conf/(local|bblayers).conf', queryset_with_search.filter(vhistory__file_name__contains='conf/local.conf').count(), 'Select this filter to see variables set by the <code>local.conf</code> and <code>bblayers.conf</code> configuration files inside the <code>/build/conf/</code> directory'),
+                               ('Local configuration variables', 'vhistory__file_name__regex:conf/(local|bblayers).conf', queryset_with_search.filter(vhistory__file_name__regex='conf/(local|bblayers).conf').count(), 'Select this filter to see variables set by the <code>local.conf</code> and <code>bblayers.conf</code> configuration files inside the <code>/build/conf/</code> directory'),
                                ('Machine configuration variables', 'vhistory__file_name__contains:conf/machine/',queryset_with_search.filter(vhistory__file_name__contains='conf/machine').count(), 'Select this filter to see variables set by the configuration file(s) inside your layers <code>/conf/machine/</code> directory'),
                                ('Distro configuration variables', 'vhistory__file_name__contains:conf/distro/',queryset_with_search.filter(vhistory__file_name__contains='conf/distro').count(), 'Select this filter to see variables set by the configuration file(s) inside your layers <code>/conf/distro/</code> directory'),
                                ('Layer configuration variables', 'vhistory__file_name__contains:conf/layer.conf',queryset_with_search.filter(vhistory__file_name__contains='conf/layer.conf').count(), 'Select this filter to see variables set by the <code>layer.conf</code> configuration file inside your layers'),
Signature · 1 line
1.8.3.2
#9 lore raw

PATCH 9/18 toaster: Change "0 found" to "No found"

In reply to Alex DAMIAN

From: Belen Barros Pena <[email protected]>

Change the string in the h1 when search returns no results from "0 things found" to "None things found". The change applies to the BitBake variables, tasks, recipes, packages built, time, CPU and disk I/O tables.

[YOCTO #5981]

---
 lib/toaster/toastergui/templates/bpackage.html   | 6 +++---
 lib/toaster/toastergui/templates/configvars.html | 6 +++---
 lib/toaster/toastergui/templates/recipes.html    | 6 +++---
 lib/toaster/toastergui/templates/tasks.html      | 6 +++---
 4 files changed, 12 insertions(+), 12 deletions(-)
lib/toaster/toastergui/templates/bpackage.html +3 -3
index 7da8ce7..dad0d7f 100644
@@ -10,10 +10,10 @@
 <div class="span10">
 <div class="page-header">
 <h1>
-  {% if request.GET.filter or request.GET.search and objects.count > 0  %}
+  {% if request.GET.search and objects.paginator.count > 0  %}
       {{objects.paginator.count}} package{{objects.paginator.count|pluralize}} found
-  {%elif objects.paginator.count == 0%}
-      No Packages
+  {%elif request.GET.search and objects.paginator.count == 0%}
+      No packages found
   {%else%}
       Packages
   {%endif%}
lib/toaster/toastergui/templates/configvars.html +3 -3
index d705253..24cb2a5 100644
@@ -10,10 +10,10 @@
 <div class="row-fluid span10">
  <div class="page-header">
  <h1>
-  {% if request.GET.filter or request.GET.search and objects.count > 0  %}
+  {% if request.GET.filter and objects.paginator.count > 0 or request.GET.search and objects.paginator.count > 0 %}
       {{objects.paginator.count}} variable{{objects.paginator.count|pluralize}} found
-  {%elif objects.paginator.count == 0%}
-      No variables
+  {%elif request.GET.filter and objects.paginator.count == 0 or request.GETs.search and objects.paginator.count == 0 %}
+      No variables found
   {%else%}
       Configuration
   {%endif%}
lib/toaster/toastergui/templates/recipes.html +3 -3
index 860d30d..22c3d1a 100755
@@ -10,10 +10,10 @@
 <div class="span10">
 <div class="page-header">
 <h1>
-  {% if request.GET.filter or request.GET.search and objects.count > 0  %}
+  {% if request.GET.search and objects.paginator.count > 0  %}
       {{objects.paginator.count}} recipe{{objects.paginator.count|pluralize}} found
-  {%elif objects.paginator.count == 0%}
-      No Recipes
+  {%elif request.GET.search and objects.paginator.count == 0%}
+      No recipes found
   {%else%}
       Recipes
   {%endif%}
lib/toaster/toastergui/templates/tasks.html +3 -3
index d881811..a2629dc 100644
@@ -8,10 +8,10 @@
 <div class="span10">
 <div class="page-header">
 <h1>
-  {% if request.GET.filter or request.GET.search and objects.count > 0  %}
+  {% if request.GET.filter or request.GET.search and objects.paginator.count > 0  %}
       {{objects.paginator.count}} task{{objects.paginator.count|pluralize}} found
-  {%elif objects.paginator.count == 0%}
-      No tasks
+  {%elif request.GET.filter or request.GET.search and objects.paginator.count == 0%}
+      No tasks found
   {%else%}
       {{title}}
   {%endif%}
Signature · 1 line
1.8.3.2
#10 lore raw

PATCH 10/18 toaster: apply filter only on selected attributes

In reply to Alex DAMIAN

From: Marius Avram <[email protected]>

This fixes the filter from the main page: 'All builds' applied to the table header. Once one is selected, the filter button for that respective column will receive a title attribute which will show additional options when you hover over it. Until now the additional option was active for all filters buttons not only for the one selected. This also caused malformed output in the case of the buttons which weren't among the selected filter.

[YOCTO #5929]

---
 lib/toaster/toastergui/templates/basetable_top.html | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/toaster/toastergui/templates/basetable_top.html +1 -1
index 58b55b6..3fc0f31 100644
@@ -77,7 +77,7 @@
                 {%if tc.orderfield%}<a {%if tc.ordericon%} class="sorted" {%endif%}href="javascript:reload_params({'page': 1, 'orderby' : '{{tc.orderfield}}' })" >{{tc.name}}</a>{%else%}<span class="muted">{{tc.name}}</span>{%endif%}
                 {%if tc.ordericon%} <i class="icon-caret-{{tc.ordericon}}"></i>{%endif%}
                 {%if tc.filter%}<div class="btn-group pull-right">
-                    <a href="#filter_{{tc.filter.class}}" role="button" class="btn btn-mini {%if request.GET.filter%}{{tc.filter.options|filtered_icon:request.GET.filter}} {%endif%}" {%if request.GET.filter%} title="<p>{{tc.filter.options|filtered_tooltip:request.GET.filter}}</p><p><a class='btn btn-small btn-primary' href=javascript:reload_params({'filter':''})>Show all {% if filter_search_display %}{{filter_search_display}}{% else %}{{objectname}}{% endif %}</a></p>" {%endif%} data-toggle="modal"> <i class="icon-filter filtered"></i> </a>
+                    <a href="#filter_{{tc.filter.class}}" role="button" class="btn btn-mini {%if request.GET.filter%}{{tc.filter.options|filtered_icon:request.GET.filter}} {%endif%}" {%if request.GET.filter and tc.filter.options|filtered_tooltip:request.GET.filter %} title="<p>{{tc.filter.options|filtered_tooltip:request.GET.filter}}</p><p><a class='btn btn-small btn-primary' href=javascript:reload_params({'filter':''})>Show all {% if filter_search_display %}{{filter_search_display}}{% else %}{{objectname}}{% endif %}</a></p>" {%endif%} data-toggle="modal"> <i class="icon-filter filtered"></i> </a>
                 </div>{%endif%}
             </th>{% endfor %}
         </tr>
Signature · 1 line
1.8.3.2
#11 lore raw

PATCH 11/18 bitbake toaster: check the file_name with the content of the IMAGE_FSTYPES variable

In reply to Alex DAMIAN

From: Cristiana Voicu <[email protected]>

File_name information of Target_image_file is being collected for a .rootfs.manifest file. We would like not to collect this. The solution is to cross check the information gathered for file_name with the content of the IMAGE_FSTYPES variable. If any of the file_name entries does not match the content of IMAGE_FSTYPES, we do not store it.

[YOCTO #5189]

---
 lib/bb/ui/buildinfohelper.py | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
lib/bb/ui/buildinfohelper.py +2 -1
index 08d9fcf..30c9b92 100644
@@ -615,11 +615,12 @@ class BuildInfoHelper(object):
         self.orm_wrapper.save_build_variables(build_obj, self.server.runCommand(["getAllKeysWithFlags", ["doc", "func"]])[0])
 
     def update_target_image_file(self, event):
+        image_fstypes = self.server.runCommand(["getVariable", "IMAGE_FSTYPES"])[0]
         for t in self.internal_state['targets']:
             if t.is_image == True:
                 output_files = list(event.data.viewkeys())
                 for output in output_files:
-                    if t.target in output:
+                    if t.target in output and output.split('.rootfs.')[1] in image_fstypes:
                         self.orm_wrapper.save_target_image_file_information(t, output, event.data[output])
 
     def update_build_information(self, event, errors, warnings, taskfailures):
Signature · 1 line
1.8.3.2
#12 lore raw

PATCH 12/18 toaster: add search by section to all recipe page

In reply to Alex DAMIAN

From: David Reyna <[email protected]>

Add the 'section' to the searchable columns list for the recipe model.

[YOCTO #5993]

---
 lib/toaster/orm/models.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/toaster/orm/models.py +1 -1
index c5fe69b..bb018a9 100644
@@ -261,7 +261,7 @@ class Package_File(models.Model):
     size = models.IntegerField()
 
 class Recipe(models.Model):
-    search_allowed_fields = ['name', 'version', 'file_path', 'license', 'layer_version__layer__name', 'layer_version__branch', 'layer_version__commit', 'layer_version__layer__local_path']
+    search_allowed_fields = ['name', 'version', 'file_path', 'section', 'license', 'layer_version__layer__name', 'layer_version__branch', 'layer_version__commit', 'layer_version__layer__local_path']
     name = models.CharField(max_length=100, blank=True)
     version = models.CharField(max_length=100, blank=True)
     layer_version = models.ForeignKey('Layer_Version', related_name='recipe_layer_version')
Signature · 1 line
1.8.3.2
#13 lore raw

PATCH 13/18 toaster: combine ready functions

In reply to Alex DAMIAN

From: Amit Kumar Chaudhary <[email protected]>

Merge the code into single ready function.

[YOCTO #4283]

---
 lib/toaster/toastergui/static/js/main.js | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
lib/toaster/toastergui/static/js/main.js +1 -3
index 07aba3d..7f897d2 100644
@@ -90,9 +90,6 @@ $(document).ready(function() {
         });
     });
 
-});
-
-$(document).ready(function() {
     //toggle the errors and warnings sections
     $('.show-errors').click(function() {
         $('#collapse-errors').addClass('in');
@@ -110,4 +107,5 @@ $(document).ready(function() {
     if (location.href.search('#warnings') > -1) {
         $('#collapse-warnings').addClass('in');
     }
+
 });
Signature · 1 line
1.8.3.2
#14 lore raw

PATCH 14/18 toaster: Remove circular dependecies from packages/recipes

In reply to Alex DAMIAN

From: Irina Patru <[email protected]>

The RelatedManager for Package_Dependency and Recipe_Dependency is changed so that circular dependacies are hidden in the UI, but will still remain in the database.

[YOCTO #5655]

---
 lib/toaster/orm/models.py | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
lib/toaster/orm/models.py +14 -0
index bb018a9..3059908 100644
@@ -20,6 +20,7 @@
 # 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
 
 from django.db import models
+from django.db.models import F
 from django.utils.encoding import python_2_unicode_compatible
 
 
@@ -211,6 +212,12 @@ class Package(models.Model):
     section = models.CharField(max_length=80, blank=True)
     license = models.CharField(max_length=80, blank=True)
 
+class Package_DependencyManager(models.Manager):
+    use_for_related_fields = True
+
+    def get_query_set(self):
+        return super(Package_DependencyManager, self).get_query_set().exclude(package_id = F('depends_on__id'))
+
 class Package_Dependency(models.Model):
     TYPE_RDEPENDS = 0
     TYPE_TRDEPENDS = 1
@@ -250,6 +257,7 @@ class Package_Dependency(models.Model):
     depends_on = models.ForeignKey(Package, related_name='package_dependencies_target')   # soft dependency
     dep_type = models.IntegerField(choices=DEPENDS_TYPE)
     target = models.ForeignKey(Target, null=True)
+    objects = Package_DependencyManager()
 
 class Target_Installed_Package(models.Model):
     target = models.ForeignKey(Target)
@@ -273,6 +281,11 @@ class Recipe(models.Model):
     bugtracker = models.URLField(blank=True)
     file_path = models.FilePathField(max_length=255)
 
+class Recipe_DependencyManager(models.Manager):
+    use_for_related_fields = True
+
+    def get_query_set(self):
+        return super(Recipe_DependencyManager, self).get_query_set().exclude(recipe_id = F('depends_on__id'))
 
 class Recipe_Dependency(models.Model):
     TYPE_DEPENDS = 0
@@ -285,6 +298,7 @@ class Recipe_Dependency(models.Model):
     recipe = models.ForeignKey(Recipe, related_name='r_dependencies_recipe')
     depends_on = models.ForeignKey(Recipe, related_name='r_dependencies_depends')
     dep_type = models.IntegerField(choices=DEPENDS_TYPE)
+    objects = Recipe_DependencyManager()
 
 class Layer(models.Model):
     name = models.CharField(max_length=100)
Signature · 1 line
1.8.3.2
#15 lore raw

PATCH 15/18 toaster: select recipe based on PN name

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

When saving task stats, if there were multiple tasks executed based on the same recipe file, we might have saved the stats to the wrong task by selecting another recipe.

This patch takes the PN into account to properly select the file stats.

A check is also made to make sure we don't fail saving data due to interrupted builds.

---
 lib/bb/ui/buildinfohelper.py | 26 ++++++++++++++------------
 1 file changed, 14 insertions(+), 12 deletions(-)
lib/bb/ui/buildinfohelper.py +14 -12
index 30c9b92..71b2ff5 100644
@@ -678,20 +678,18 @@ class BuildInfoHelper(object):
 
 
     def store_tasks_stats(self, event):
-        for (taskfile, taskname, taskstats) in event.data:
+        for (taskfile, taskname, taskstats, recipename) in event.data:
             localfilepath = taskfile.split(":")[-1]
             assert localfilepath.startswith("/")
 
             recipe_information = self._get_recipe_information_from_taskfile(taskfile)
-            try:
-                recipe = self.orm_wrapper.get_update_recipe_object(recipe_information, True)
-            except NotExisting:
-                recipe = Recipe.objects.get(layer_version = recipe_information['layer_version'],
-                            file_path__endswith = recipe_information['file_path'])
+            recipe_object = Recipe.objects.get(layer_version = recipe_information['layer_version'],
+                            file_path__endswith = recipe_information['file_path'],
+                            name = recipename)
 
             task_information = {}
             task_information['build'] = self.internal_state['build']
-            task_information['recipe'] = recipe
+            task_information['recipe'] = recipe_object
             task_information['task_name'] = taskname
             task_information['cpu_usage'] = taskstats['cpu_usage']
             task_information['disk_io'] = taskstats['disk_io']
@@ -756,11 +754,15 @@ class BuildInfoHelper(object):
         # for all image targets
         for target in self.internal_state['targets']:
             if target.is_image:
-                pkgdata = event.data['pkgdata']
-                imgdata = event.data['imgdata'][target.target]
-                self.orm_wrapper.save_target_package_information(self.internal_state['build'], target, imgdata, pkgdata, self.internal_state['recipes'])
-                filedata = event.data['filedata'][target.target]
-                self.orm_wrapper.save_target_file_information(self.internal_state['build'], target, filedata)
+                try:
+                    pkgdata = event.data['pkgdata']
+                    imgdata = event.data['imgdata'][target.target]
+                    self.orm_wrapper.save_target_package_information(self.internal_state['build'], target, imgdata, pkgdata, self.internal_state['recipes'])
+                    filedata = event.data['filedata'][target.target]
+                    self.orm_wrapper.save_target_file_information(self.internal_state['build'], target, filedata)
+                except KeyError:
+                    # we must have not got the data for this image, nothing to save
+                    pass
 
 
 
Signature · 1 line
1.8.3.2
#16 lore raw

PATCH 16/18 toasterui: save missed sstate tasks

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

We save the missed sstate tasks as tasks that executed but have the sstate_result set to "SSTATE_MISSED", signaling that the attempt to find an sstate file failed.

---
 lib/bb/ui/buildinfohelper.py | 21 +++++++++++++++++++++
 lib/bb/ui/toasterui.py       |  2 ++
 2 files changed, 23 insertions(+)
lib/bb/ui/buildinfohelper.py +21 -0
index 71b2ff5..d1d92c8 100644
@@ -749,6 +749,27 @@ class BuildInfoHelper(object):
         self.orm_wrapper.get_update_task_object(task_information, True) # must exist
 
 
+    def store_missed_state_tasks(self, event):
+        for (fn, taskname, taskhash, sstatefile) in event.data:
+
+            identifier = fn + taskname + "_setscene"
+            recipe_information = self._get_recipe_information_from_taskfile(fn)
+            recipe = self.orm_wrapper.get_update_recipe_object(recipe_information)
+            class MockEvent: pass
+            event = MockEvent()
+            event.taskname = taskname
+            event.taskhash = taskhash
+            task_information = self._get_task_information(event,recipe)
+
+            task_information['start_time'] = datetime.datetime.now()
+            task_information['outcome'] = Task.OUTCOME_NA
+            task_information['sstate_checksum'] = taskhash
+            task_information['sstate_result'] = Task.SSTATE_MISS
+            task_information['path_to_sstate_obj'] = sstatefile
+
+            self.orm_wrapper.get_update_task_object(task_information)
+
+
     def store_target_package_data(self, event):
         assert 'data' in vars(event)
         # for all image targets
lib/bb/ui/toasterui.py +2 -0
index b2c084c..453eaf9 100644
@@ -241,6 +241,8 @@ def main(server, eventHandler, params ):
                     buildinfohelper.store_tasks_stats(event)
                 elif event.type == "ImagePkgList":
                     buildinfohelper.store_target_package_data(event)
+                elif event.type == "MissedSstate":
+                    buildinfohelper.store_missed_state_tasks(event)
                 elif event.type == "ImageFileSize":
                     buildinfohelper.update_target_image_file(event)
                 elif event.type == "LicenseManifestPath":
Signature · 1 line
1.8.3.2
#17 lore raw

PATCH 17/18 toaster: clean exit on bb server shutdown

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

This patch adds the capability to have the Toaster UI detect when the Bitbake server exited and cleanly trigger a clean shutdown of the system through the toaster starting script.

---
 bin/toaster            | 28 ++++++++++++++++++++++++++--
 lib/bb/ui/toasterui.py |  7 +++++--
 2 files changed, 31 insertions(+), 4 deletions(-)
bin/toaster +26 -2
index 9c0a15f..b27f7c2 100755
@@ -68,11 +68,16 @@ function addtoConfiguration()
         echo $1 >> ${BUILDDIR}/conf/$2
 }
 
+INSTOPSYSTEM=0
+
 # define the stop command
 function stop_system()
 {
+    # prevent reentry
+    if [ $INSTOPSYSTEM == 1 ]; then return; fi
+    INSTOPSYSTEM=1
     if [ -f ${BUILDDIR}/.toasterui.pid ]; then
-        kill $(< ${BUILDDIR}/.toasterui.pid )
+        kill $(< ${BUILDDIR}/.toasterui.pid ) 2>/dev/null
         rm ${BUILDDIR}/.toasterui.pid
     fi
     BBSERVER=localhost:8200 bitbake -m
@@ -80,8 +85,24 @@ function stop_system()
     webserverKillAll
     # force stop any misbehaving bitbake server
     lsof bitbake.lock | awk '{print $2}' | grep "[0-9]\+" | xargs -n1 -r kill
+    trap - SIGHUP
+    trap - SIGCHLD
+    INSTOPSYSTEM=0
+}
+
+function check_pidbyfile() {
+    [ -e $1 ] && kill -0 $(< $1) 2>/dev/null
 }
 
+
+function notify_chldexit() {
+    if [ $NOTOASTERUI == 0 ]; then
+        check_pidbyfile ${BUILDDIR}/.toasterui.pid && return
+        stop_system
+    fi
+}
+
+
 # We make sure we're running in the current shell and in a good environment
 
 if [ -z "$ZSH_NAME" ] && [ `basename \"$0\"` = `basename \"$BASH_SOURCE\"` ]; then
@@ -179,10 +200,13 @@ case $CMD in
             stop_system
             echo "Failed ${CMD}."
         fi
+        # stop system on terminal exit
+        set -o monitor
+        trap stop_system SIGHUP
+        trap notify_chldexit SIGCHLD
     ;;
     stop )
         stop_system
-        trap '' SIGHUP
         echo "Successful ${CMD}."
     ;;
 esac
lib/bb/ui/toasterui.py +5 -2
index 453eaf9..37d6b1a 100644
@@ -249,6 +249,10 @@ def main(server, eventHandler, params ):
                     buildinfohelper.store_license_manifest_path(event)
                 continue
 
+            if isinstance(event, bb.cooker.CookerExit):
+                # exit when the server exits
+                break
+
             # ignore
             if isinstance(event, (bb.event.BuildBase,
                                   bb.event.StampUpdate,
@@ -258,8 +262,7 @@ def main(server, eventHandler, params ):
                                   bb.event.OperationProgress,
                                   bb.command.CommandFailed,
                                   bb.command.CommandExit,
-                                  bb.command.CommandCompleted,
-                                  bb.cooker.CookerExit)):
+                                  bb.command.CommandCompleted)):
                 continue
 
             if isinstance(event, bb.event.DepTreeGenerated):
Signature · 1 line
1.8.3.2
#18 lore raw

PATCH 18/18 toaster: replace package dependency tag w/ view queries

In reply to Alex DAMIAN

From: Alexandru DAMIAN <[email protected]>

This patch moves the queries for package dependencies from the project tags to the views. This is done to bring the code inline with the Django philosophy of making all data queries in views.py

This change has no performance implication.

---
 lib/toaster/toastergui/templates/target.html       |  4 ++--
 lib/toaster/toastergui/templatetags/projecttags.py | 14 --------------
 lib/toaster/toastergui/views.py                    |  6 ++++++
 3 files changed, 8 insertions(+), 16 deletions(-)
lib/toaster/toastergui/templates/target.html +2 -2
index 4512898..5396b17 100644
@@ -79,7 +79,7 @@
             {{package.license}}
         </td>
         <td class="depends">
-            {% with deps=package|runtime_dependencies:target.id %}
+            {% with deps=package.runtime_dependencies %}
             {% with deps_count=deps|length %}
             {% if deps_count > 0 %}
             <a class="btn"
@@ -96,7 +96,7 @@
             {% endwith %}
         </td>
         <td class="brought_in_by">
-            {% with rdeps=package|reverse_runtime_dependencies:target.id %}
+            {% with rdeps=package.reverse_runtime_dependencies %}
             {% with rdeps_count=rdeps|length %}
             {% if rdeps_count > 0 %}
             <a class="btn"
lib/toaster/toastergui/templatetags/projecttags.py +0 -14
index 39e92ce..12c2468 100644
@@ -217,20 +217,6 @@ def filtered_packageversion(version, revision):
     """
     return "" if (not version or version == "") else version if (not revision or revision == "") else version + "-" + revision
         
-from django.db import models
-from orm.models import Package
[email protected]
-def runtime_dependencies(package_object, targetid):
-    """ Return a queryset that lists the packages this package depends on
-    """
-    return package_object.package_dependencies_source.filter(target_id__exact=targetid, dep_type__in={'1'})
-
[email protected]
-def reverse_runtime_dependencies(package_object, targetid):
-    """ Return a queryset that lists the packages depending on this package
-    """
-    return package_object.package_dependencies_target.filter(target_id__exact = targetid,dep_type__in={'1'})
-
 @register.filter
 def filter_sizeovertotal(package_object, total_size):
     """ Return the % size of the package over the total size argument
lib/toaster/toastergui/views.py +6 -0
index 732be8f..e523c31 100644
@@ -429,6 +429,12 @@ def target(request, build_id, target_id):
     packages_sum =  queryset.aggregate(Sum('installed_size'))
     queryset = _get_queryset(Package, queryset, filter_string, search_term, ordering_string)
     packages = _build_page_range(Paginator(queryset, request.GET.get('count', 25)),request.GET.get('page', 1))
+
+    # bring in package dependencies
+    for p in packages.object_list:
+        p.runtime_dependencies = p.package_dependencies_source.filter(target_id = target_id, dep_type=Package_Dependency.TYPE_TRDEPENDS)
+        p.reverse_runtime_dependencies = p.package_dependencies_target.filter(target_id = target_id, dep_type=Package_Dependency.TYPE_TRDEPENDS)
+
     context = { 'build': Build.objects.filter(pk=build_id)[0],
                 'target': Target.objects.filter(pk=target_id)[0],
                 'objects': packages,
Signature · 1 line
1.8.3.2