Skip to contentSkip to conversation

update.py: fix Ctrl+C behaviour

PATCH New Paul Eggleton 1 participant 6 messages Feb 20, 2018

lore
Share & tools
lore
#0 lore raw

PATCH 1/6 update.py: fix Ctrl+C behaviour

If the user hit Ctrl+C during the initial info gathering then it didn't break out of the loop in update.py, so you had to hit Ctrl+C for as many layers as were involved in the update. Look for exit code 254 from update_layer.py and stop if it is returned since that indicates Ctrl+C has been used.

Additionally, ensure we return exit code 254 and print a message from the main update script when it is interrupted in this way.

---
 layerindex/update.py | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
layerindex/update.py +10 -2
index 5c83b00..f60b943 100755
@@ -337,7 +337,11 @@ def main():
                     logger.debug('Running layer update command: %s' % cmd)
                     ret, output = run_command_interruptible(cmd)
                     logger.debug('output: %s' % output)
-                    if ret != 0:
+                    if ret == 254:
+                        # Interrupted by user, break out of loop
+                        logger.info('Update interrupted, exiting')
+                        sys.exit(254)
+                    elif ret != 0:
                         continue
                     col = re.search("^BBFILE_COLLECTIONS = \"(.*)\"", output, re.M).group(1) or ''
                     ver = re.search("^LAYERVERSION = \"(.*)\"", output, re.M).group(1) or ''
@@ -418,10 +422,14 @@ def main():
 
                     if ret == 254:
                         # Interrupted by user, break out of loop
-                        break
+                        logger.info('Update interrupted, exiting')
+                        sys.exit(254)
         finally:
             utils.unlock_file(lockfile)
 
+    except KeyboardInterrupt:
+        logger.info('Update interrupted, exiting')
+        sys.exit(254)
     finally:
         update.log = ''.join(listhandler.read())
         update.finished = datetime.now()
Signature · 1 line
2.14.3
#1 lore raw

PATCH 2/6 Add statistics page

Add a page with basic statistics for the index - number of layers, recipes, classes, machines and distros on an overall basis (distinct names) and per branch, since I've been asked a few times for this kind of information. It's currently only linked from the Tools menu for logged-in users, but the URL will work for anyone.

---
 layerindex/urls.py              |  6 ++++-
 layerindex/views.py             | 17 ++++++++++++
 templates/base.html             |  1 +
 templates/layerindex/stats.html | 60 +++++++++++++++++++++++++++++++++++++++++
 4 files changed, 83 insertions(+), 1 deletion(-)
 create mode 100644 templates/layerindex/stats.html
layerindex/urls.py +5 -1
index 7deaf23..d2f9eab 100644
@@ -8,7 +8,7 @@ from django.conf.urls import *
 from django.views.generic import TemplateView, DetailView, ListView, RedirectView
 from django.views.defaults import page_not_found
 from django.core.urlresolvers import reverse_lazy
-from layerindex.views import LayerListView, LayerReviewListView, LayerReviewDetailView, RecipeSearchView, MachineSearchView, PlainTextListView, LayerDetailView, edit_layer_view, delete_layer_view, edit_layernote_view, delete_layernote_view, HistoryListView, EditProfileFormView, AdvancedRecipeSearchView, BulkChangeView, BulkChangeSearchView, bulk_change_edit_view, bulk_change_patch_view, BulkChangeDeleteView, RecipeDetailView, RedirectParamsView, ClassicRecipeSearchView, ClassicRecipeDetailView, ClassicRecipeStatsView, LayerUpdateDetailView, UpdateListView, UpdateDetailView
+from layerindex.views import LayerListView, LayerReviewListView, LayerReviewDetailView, RecipeSearchView, MachineSearchView, PlainTextListView, LayerDetailView, edit_layer_view, delete_layer_view, edit_layernote_view, delete_layernote_view, HistoryListView, EditProfileFormView, AdvancedRecipeSearchView, BulkChangeView, BulkChangeSearchView, bulk_change_edit_view, bulk_change_patch_view, BulkChangeDeleteView, RecipeDetailView, RedirectParamsView, ClassicRecipeSearchView, ClassicRecipeDetailView, ClassicRecipeStatsView, LayerUpdateDetailView, UpdateListView, UpdateDetailView, StatsView
 from layerindex.models import LayerItem, Recipe, RecipeChangeset
 from rest_framework import routers
 from . import restviews
@@ -127,6 +127,10 @@ urlpatterns = patterns('',
         TemplateView.as_view(
             template_name='layerindex/about.html'),
             name="about"),
+    url(r'^stats/$',
+        StatsView.as_view(
+            template_name='layerindex/stats.html'),
+            name='stats'),
     url(r'^oe-classic/$',
         RedirectView.as_view(url=reverse_lazy('classic_recipe_search'), permanent=False),
             name='classic'),
layerindex/views.py +17 -0
index 4f6c2c9..95812ca 100644
@@ -1007,3 +1007,20 @@ class ClassicRecipeStatsView(TemplateView):
             'jquery_on_ready': False,
         }
         return context
+
+
+class StatsView(TemplateView):
+    def get_context_data(self, **kwargs):
+        context = super(StatsView, self).get_context_data(**kwargs)
+        context['layercount'] = LayerItem.objects.count()
+        context['recipe_count_distinct'] = Recipe.objects.values('pn').distinct().count()
+        context['class_count_distinct'] = BBClass.objects.values('name').distinct().count()
+        context['machine_count_distinct'] = Machine.objects.values('name').distinct().count()
+        context['distro_count_distinct'] = Distro.objects.values('name').distinct().count()
+        context['perbranch'] = Branch.objects.order_by('sort_priority').annotate(
+                layer_count=Count('layerbranch', distinct=True),
+                recipe_count=Count('layerbranch__recipe', distinct=True),
+                class_count=Count('layerbranch__bbclass', distinct=True),
+                machine_count=Count('layerbranch__machine', distinct=True),
+                distro_count=Count('layerbranch__distro', distinct=True))
+        return context
templates/base.html +1 -0
index 8a3b8fe..a9d7c83 100644
@@ -73,6 +73,7 @@
                             <li><a href="{% url 'bulk_change' %}">Bulk Change</a></li>
                             <li><a href="{% url 'duplicates' 'master' %}">Duplicates</a></li>
                             <li><a href="{% url 'update_list' %}">Updates</a></li>
+                            <li><a href="{% url 'stats' %}">Statistics</a></li>
                         </ul>
                     </li>
                     {% endif %}
templates/layerindex/stats.html new file +60 -0

60 lines not on this page · raw message

Signature · 1 line
2.14.3
#2 lore raw

PATCH 3/6 Show layer description with newlines in layer detail

In reply to Paul Eggleton

A lot of people enter line breaks in their layer descriptions hoping that these will show up in the layer description, but of course since they are being displayed as part of an HTML document, they don't by default. Use a style in the description paragraph to ensure that they do.

---
 templates/layerindex/detail.html | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
templates/layerindex/detail.html +1 -3
index 0747421..3081ef3 100644
@@ -76,9 +76,7 @@
 
             <div class="row-fluid">
                 <div class="description span7">
-                    <p>
-                       {{ layeritem.description }}
-                    </p>
+                    <p style="white-space: pre">{{ layeritem.description }}</p>
                     <p>
                         {% if layeritem.usage_url %}
                             <span class="label label-info">
Signature · 1 line
2.14.3
#3 lore raw

PATCH 4/6 requirements.txt: add missing dependencies

In reply to Paul Eggleton

These are dependencies of items already listed in requirements.txt, so nothing new.

---
 requirements.txt | 5 +++++
 1 file changed, 5 insertions(+)
requirements.txt +5 -0
index 7cd741b..af6ab5a 100644
@@ -1,3 +1,6 @@
+amqp==1.4.9
+anyjson==0.3.3
+billiard==3.3.0.23
 celery==3.1.25
 Django==1.8.17
 django-cors-headers==1.1.0
@@ -10,11 +13,13 @@ djangorestframework==3.2.5
 gitdb==0.6.4
 GitPython==2.0.5
 Jinja2==2.8
+kombu==3.0.37
 MarkupSafe==0.23
 mysqlclient==1.3.7
 Pillow==3.3.0
 python-nvd3==0.14.2
 python-slugify==1.1.4
+pytz==2018.3
 six==1.10.0
 smmap==0.9.0
 Unidecode==0.4.19
Signature · 1 line
2.14.3
#4 lore raw

PATCH 5/6 requirements.txt: update some dependency versions

In reply to Paul Eggleton

Use latest tested versions (though we pin djangorestframework at 3.6.4 since that is the last version that supports Django 1.8), and add new resulting dependencies.

---
 requirements.txt | 34 +++++++++++++++++++---------------
 1 file changed, 19 insertions(+), 15 deletions(-)
requirements.txt +19 -15
index af6ab5a..58100a9 100644
@@ -1,25 +1,29 @@
-amqp==1.4.9
+amqp==2.2.2
 anyjson==0.3.3
-billiard==3.3.0.23
-celery==3.1.25
-Django==1.8.17
+billiard==3.5.0.3
+celery==4.1.0
+confusable-homoglyphs==2.0.2
+Django==1.8.18
 django-cors-headers==1.1.0
 django-nvd3==0.9.7
-django-registration==2.1
+django-ranged-response==0.2.0
+django-registration==2.4.1
 django-reversion==1.9.3
 django-reversion-compare==0.5.6
-django-simple-captcha==0.4.6
-djangorestframework==3.2.5
+django-simple-captcha==0.5.6
+djangorestframework==3.6.4
 gitdb==0.6.4
-GitPython==2.0.5
-Jinja2==2.8
-kombu==3.0.37
-MarkupSafe==0.23
-mysqlclient==1.3.7
-Pillow==3.3.0
+GitPython==2.1.8
+Jinja2==2.10
+kombu==4.1.0
+MarkupSafe==1.0
+mysqlclient==1.3.12
+Pillow==5.0.0
 python-nvd3==0.14.2
-python-slugify==1.1.4
+python-slugify==1.2.4
 pytz==2018.3
-six==1.10.0
+six==1.11.0
 smmap==0.9.0
+smmap2==2.0.3
 Unidecode==0.4.19
+vine==1.1.4
Signature · 1 line
2.14.3
#5 lore raw

PATCH 6/6 README: update dependency versions

In reply to Paul Eggleton

Update the minor version of Django, and replace the list of dependencies with a pointer to requirements.txt since there's not much point maintaining that in two places (and the README wasn't complete anyway).

---
 README | 13 ++++---------
 1 file changed, 4 insertions(+), 9 deletions(-)
README +4 -9
index c7f7409..970c267 100644
@@ -12,7 +12,7 @@ Setup
 In order to make use of this application you will need:
 
 * Python 3.4+
-* Django 1.8.x - tested with 1.8.17; newer versions may work, but
+* Django 1.8.x - tested with 1.8.18; newer versions may work, but
   the application has not been tested with 1.9 or newer.
 * RabbitMQ 3.6.x - tested with 3.6.10.
 * For production usage, a web server set up to host Django applications
@@ -20,14 +20,9 @@ In order to make use of this application you will need:
 * A database supported by Django (SQLite, MySQL, etc.). Django takes
   care of creating the database itself, you just need to ensure that the
   database server (if not using SQLite) is configured and running.
-* The following third-party Django modules (tested versions listed):
-  * django-registration (2.1)
-  * django-reversion (1.9.3)
-  * django-reversion-compare (0.5.6)
-  * django-simple-captcha (0.4.6)
-  * django-nvd3 (0.14.2)
-  * djangorestframework (3.2.5)
-  * django-cors-headers (1.1.0)
+* Third-party Django/python modules as detailed by requirements.txt.
+  If you're using pip, you can install these by running:
+    pip install -r requirements.txt
 * On the machine that will run the backend update script (which does not
   have to be the same machine as the web server, however it does still
   have to have Django installed, have the same or similar configuration
Signature · 1 line
2.14.3