Skip to contentSkip to conversation

toaster_cummulative_08282023

PATCH In review David Reyna 2 participants 5 messages Aug 28, 2023 – Aug 29, 2023

lore
Share & tools
lore
#0 lore raw

cover toaster_cummulative_08282023

toaster_cummulative_08282023

The following changes since commit c8e9590a376c0b63e38167dc522fac4cfbecb5d4:

bitbake: Fix disk space monitoring on cephfs (2023-08-25 07:55:34 +0100)

are available in the Git repository at:

ssh://[email protected]/poky-contrib dreyna/submit/dreyna/toaster/toaster_cummulative_08282023

David Reyna (3):
toaster: Update to Django 4.2
toaster: import only used layers
toaster: accommodate missing 'Image Name' value in buildinfohelper

 lib/bb/ui/buildinfohelper.py                  | 13 ++++++++-
 lib/toaster/bldcollector/urls.py              |  2 +-
 lib/toaster/bldcontrol/models.py              |  4 +--
 lib/toaster/orm/models.py                     | 28 +++++++++----------
 lib/toaster/toastergui/urls.py                |  2 +-
 lib/toaster/toastergui/views.py               |  6 ++--
 lib/toaster/toastergui/widgets.py             |  5 ++--
 .../management/commands/buildimport.py        |  2 +-
 .../management/commands/checksocket.py        |  4 +--
 lib/toaster/toastermain/urls.py               |  2 +-
 toaster-requirements.txt                      |  2 +-
 11 files changed, 41 insertions(+), 29 deletions(-)
Signature · 1 line
2.20.1
#1 lore raw

PATCH 1/3 toaster: Update to Django 4.2

Update Toaster to support Django 4.2, to match current hosts and to address CVEs.

[YOCTO #15152]

---
 lib/toaster/bldcollector/urls.py                           | 2 +-
 lib/toaster/bldcontrol/models.py                           | 4 ++--
 lib/toaster/toastergui/urls.py                             | 2 +-
 lib/toaster/toastergui/views.py                            | 4 ++--
 lib/toaster/toastergui/widgets.py                          | 5 +++--
 lib/toaster/toastermain/management/commands/checksocket.py | 4 ++--
 lib/toaster/toastermain/urls.py                            | 2 +-
 toaster-requirements.txt                                   | 2 +-
 8 files changed, 13 insertions(+), 12 deletions(-)
lib/toaster/bldcollector/urls.py +1 -1
index efd67a81a54..3c34070351c 100644
@@ -6,7 +6,7 @@
 # SPDX-License-Identifier: GPL-2.0-only
 #
 
-from django.conf.urls import url
+from django.urls import re_path as url
 
 import bldcollector.views
 
lib/toaster/bldcontrol/models.py +2 -2
index c2f302da24a..42750e7180a 100644
@@ -4,7 +4,7 @@
 
 from __future__ import unicode_literals
 from django.db import models
-from django.utils.encoding import force_text
+from django.utils.encoding import force_str
 from orm.models import Project, Build, Layer_Version
 
 import logging
@@ -124,7 +124,7 @@ class BuildRequest(models.Model):
         return self.brvariable_set.get(name="MACHINE").value
 
     def __str__(self):
-        return force_text('%s %s' % (self.project, self.get_state_display()))
+        return force_str('%s %s' % (self.project, self.get_state_display()))
 
 # These tables specify the settings for running an actual build.
 # They MUST be kept in sync with the tables in orm.models.Project*
lib/toaster/toastergui/urls.py +1 -1
index d2df4e60488..bc3b0c79d82 100644
@@ -6,7 +6,7 @@
 # SPDX-License-Identifier: GPL-2.0-only
 #
 
-from django.conf.urls import url
+from django.urls import re_path as url
 from django.views.generic import RedirectView
 
 from toastergui import tables
lib/toaster/toastergui/views.py +2 -2
index a571b8cc18c..bf92510cdcd 100644
@@ -670,11 +670,11 @@ def xhr_dirinfo(request, build_id, target_id):
     return HttpResponse(_get_dir_entries(build_id, target_id, top), content_type = "application/json")
 
 from django.utils.functional import Promise
-from django.utils.encoding import force_text
+from django.utils.encoding import force_str
 class LazyEncoder(json.JSONEncoder):
     def default(self, obj):
         if isinstance(obj, Promise):
-            return force_text(obj)
+            return force_str(obj)
         return super(LazyEncoder, self).default(obj)
 
 from toastergui.templatetags.projecttags import filtered_filesizeformat
lib/toaster/toastergui/widgets.py +3 -2
index ceff52942ee..53696912a46 100644
@@ -7,6 +7,7 @@
 #
 
 from django.views.generic import View, TemplateView
+from django.utils.decorators import method_decorator
 from django.views.decorators.cache import cache_control
 from django.shortcuts import HttpResponse
 from django.core.cache import cache
@@ -63,8 +64,8 @@ class ToasterTable(TemplateView):
         self.default_orderby = ""
 
     # prevent HTTP caching of table data
-    @cache_control(must_revalidate=True,
-                   max_age=0, no_store=True, no_cache=True)
+    @method_decorator(cache_control(must_revalidate=True,
+                   max_age=0, no_store=True, no_cache=True))
     def dispatch(self, *args, **kwargs):
         return super(ToasterTable, self).dispatch(*args, **kwargs)
 
lib/toaster/toastermain/management/commands/checksocket.py +2 -2
index 811fd5d5161..b2c002da7aa 100644
@@ -13,7 +13,7 @@ import errno
 import socket
 
 from django.core.management.base import BaseCommand, CommandError
-from django.utils.encoding import force_text
+from django.utils.encoding import force_str
 
 DEFAULT_ADDRPORT = "0.0.0.0:8000"
 
@@ -51,7 +51,7 @@ class Command(BaseCommand):
             if hasattr(err, 'errno') and err.errno in errors:
                 errtext = errors[err.errno]
             else:
-                errtext = force_text(err)
+                errtext = force_str(err)
             raise CommandError(errtext)
 
         self.stdout.write("OK")
lib/toaster/toastermain/urls.py +1 -1
index 5fb520b384d..03603026688 100644
@@ -6,7 +6,7 @@
 # SPDX-License-Identifier: GPL-2.0-only
 #
 
-from django.conf.urls import include, url
+from django.urls import re_path as url, include
 from django.views.generic import RedirectView, TemplateView
 from django.views.decorators.cache import never_cache
 import bldcollector.views
toaster-requirements.txt +1 -1
index dedd423556a..c1f433f9ec6 100644
@@ -1,3 +1,3 @@
-Django>3.2,<3.3
+Django>4.2,<4.3
 beautifulsoup4>=4.4.0
 pytz
Signature · 1 line
2.20.1
#2 lore raw

PATCH 2/3 toaster: import only used layers

In reply to David Reyna

If you import a build directory, Toaster still adds openembedded-core, meta-poky and meta-yocto-bsp to the newly created project. Toaster should only be including in the project the layers that it imported.

[YOCTO #13764]

---
 lib/toaster/orm/models.py                     | 28 +++++++++----------
 lib/toaster/toastergui/views.py               |  2 +-
 .../management/commands/buildimport.py        |  2 +-
 3 files changed, 16 insertions(+), 16 deletions(-)
lib/toaster/orm/models.py +14 -14
index 2cb7d7e0499..f9fcf9e4fbb 100644
@@ -107,7 +107,7 @@ class ToasterSetting(models.Model):
 
 
 class ProjectManager(models.Manager):
-    def create_project(self, name, release, existing_project=None):
+    def create_project(self, name, release, existing_project=None, imported=False):
         if existing_project and (release is not None):
             prj = existing_project
             prj.bitbake_version = release.bitbake_version
@@ -134,19 +134,19 @@ class ProjectManager(models.Manager):
 
         if release is None:
             return prj
-
-        for rdl in release.releasedefaultlayer_set.all():
-            lv = Layer_Version.objects.filter(
-                layer__name=rdl.layer_name,
-                release=release).first()
-
-            if lv:
-                ProjectLayer.objects.create(project=prj,
-                                            layercommit=lv,
-                                            optional=False)
-            else:
-                logger.warning("Default project layer %s not found" %
-                               rdl.layer_name)
+        if not imported:
+            for rdl in release.releasedefaultlayer_set.all():
+                lv = Layer_Version.objects.filter(
+                    layer__name=rdl.layer_name,
+                    release=release).first()
+
+                if lv:
+                    ProjectLayer.objects.create(project=prj,
+                                                layercommit=lv,
+                                                optional=False)
+                else:
+                    logger.warning("Default project layer %s not found" %
+                                rdl.layer_name)
 
         return prj
 
lib/toaster/toastergui/views.py +1 -1
index bf92510cdcd..552ff1649bf 100644
@@ -1404,7 +1404,7 @@ if True:
                     if not os.path.isdir('%s/conf' % request.POST['importdir']):
                         raise BadParameterException("Bad path or missing 'conf' directory (%s)" % request.POST['importdir'])
                     from django.core import management
-                    management.call_command('buildimport', '--command=import', '--name=%s' % request.POST['projectname'], '--path=%s' % request.POST['importdir'], interactive=False)
+                    management.call_command('buildimport', '--command=import', '--name=%s' % request.POST['projectname'], '--path=%s' % request.POST['importdir'])
                     prj = Project.objects.get(name = request.POST['projectname'])
                     prj.merged_attr = True
                     prj.save()
lib/toaster/toastermain/management/commands/buildimport.py +1 -1
index e25b55e5ab9..f7139aa0419 100644
@@ -545,7 +545,7 @@ class Command(BaseCommand):
             # Find the directory's release, and promote to default_release if local paths
             release = self.find_import_release(layers_list,lv_dict,default_release)
             # create project, SANITY: reuse any project of same name
-            project = Project.objects.create_project(project_name,release,project)
+            project = Project.objects.create_project(project_name,release,project, imported=True)
             # Apply any new layers or variables
             self.apply_conf_variables(project,layers_list,lv_dict,release)
             # WORKAROUND: since we now derive the release, redirect 'newproject_specific' to 'project_specific'
Signature · 1 line
2.20.1
#3 lore raw

PATCH 3/3 toaster: accommodate missing 'Image Name' value in buildinfohelper

In reply to David Reyna

The value "Image Name" in bitbake events was missing for certain builds. Update 'buildinfohelper' to extract the image name elsewhere in this circumstance and not crash.

[YOCTO #13191]

---
 lib/bb/ui/buildinfohelper.py | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
lib/bb/ui/buildinfohelper.py +12 -1
index 129bb329c38..8b212b78034 100644
@@ -1746,7 +1746,6 @@ class BuildInfoHelper(object):
 
         buildname = self.server.runCommand(['getVariable', 'BUILDNAME'])[0]
         machine = self.server.runCommand(['getVariable', 'MACHINE'])[0]
-        image_name = self.server.runCommand(['getVariable', 'IMAGE_NAME'])[0]
 
         # location of the manifest files for this build;
         # note that this file is only produced if an image is produced
@@ -1767,6 +1766,18 @@ class BuildInfoHelper(object):
         # filter out anything which isn't an image target
         image_targets = [target for target in targets if target.is_image]
 
+        if len(image_targets) > 0:
+            #if there are image targets retrieve image_name
+            image_name = self.server.runCommand(['getVariable', 'IMAGE_NAME'])[0]
+            if not image_name:
+                #When build target is an image and image_name is not found as an environment variable
+                logger.info("IMAGE_NAME not found, extracting from bitbake command")
+                cmd = self.server.runCommand(['getVariable','BB_CMDLINE'])[0]
+                #filter out tokens that are command line options
+                cmd = [token for token in cmd if not token.startswith('-')]
+                image_name = cmd[1].split(':', 1)[0] # remove everything after : in image name
+                logger.info("IMAGE_NAME found as : %s " % image_name)
+
         for image_target in image_targets:
             # this is set to True if we find at least one file relating to
             # this target; if this remains False after the scan, we copy the
Signature · 1 line
2.20.1
#4 lore raw
In reply to David Reyna

Hello David,

Please set you git configuration so your patches include a From: line,
see https://www.openembedded.org/wiki/How_to_submit_a_patch_to_OpenEmbedded#Fixing_your_From_identity

Quoted 41 lines · David via lists.openembedded.org

toaster_cummulative_08282023

The following changes since commit c8e9590a376c0b63e38167dc522fac4cfbecb5d4:

bitbake: Fix disk space monitoring on cephfs (2023-08-25 07:55:34 +0100)

are available in the Git repository at:

ssh://[email protected]/poky-contrib dreyna/submit/dreyna/toaster/toaster_cummulative_08282023

David Reyna (3):
toaster: Update to Django 4.2
toaster: import only used layers
toaster: accommodate missing 'Image Name' value in buildinfohelper

 lib/bb/ui/buildinfohelper.py                  | 13 ++++++++-
 lib/toaster/bldcollector/urls.py              |  2 +-
 lib/toaster/bldcontrol/models.py              |  4 +--
 lib/toaster/orm/models.py                     | 28 +++++++++----------
 lib/toaster/toastergui/urls.py                |  2 +-
 lib/toaster/toastergui/views.py               |  6 ++--
 lib/toaster/toastergui/widgets.py             |  5 ++--
 .../management/commands/buildimport.py        |  2 +-
 .../management/commands/checksocket.py        |  4 +--
 lib/toaster/toastermain/urls.py               |  2 +-
 toaster-requirements.txt                      |  2 +-
 11 files changed, 41 insertions(+), 29 deletions(-)
Signature · 11 lines
2.20.1



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#14985): https://lists.openembedded.org/g/bitbake-devel/message/14985
Mute This Topic: https://lists.openembedded.org/mt/101006437/3617179
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/bitbake-devel/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com