Skip to contentSkip to conversation

toaster: Rationalise mimetype guessing to fix artifact downloads

PATCH In review Elliot Smith 2 participants 2 messages Sep 24, 2015 – Sep 28, 2015

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 toaster: Rationalise mimetype guessing to fix artifact downloads

Artifact download links were broken because the function to get the mimetype for the artifact was incorrectly using the underlying mimetype library. The function was also attached to the build environment controller, which was unnecessary, as we only support local controllers anyway.

Remove the mimetype getter on the build environment and use the one in the view code instead. This works correctly and prevents the download error from occurring.

[YOCTO #8369]

---
 bitbake/lib/toaster/bldcontrol/models.py | 34 --------------------------------
 bitbake/lib/toaster/toastergui/views.py  |  2 +-
 2 files changed, 1 insertion(+), 35 deletions(-)
bitbake/lib/toaster/bldcontrol/models.py +0 -34
index b61de58..c577b10 100644
@@ -39,40 +39,6 @@ class BuildEnvironment(models.Model):
     created     = models.DateTimeField(auto_now_add = True)
     updated     = models.DateTimeField(auto_now = True)
 
-
-    def get_artifact_type(self, path):
-        if self.betype == BuildEnvironment.TYPE_LOCAL:
-            try:
-                import magic
-
-                # fair warning: this is a mess; there are multiple competeing and incompatible
-                # magic modules floating around, so we try some of the most common combinations
-
-                try:    # we try ubuntu's python-magic 5.4
-                    m = magic.open(magic.MAGIC_MIME_TYPE)
-                    m.load()
-                    return m.file(path)
-                except AttributeError:
-                    pass
-
-                try:    # we try python-magic 0.4.6
-                    m = magic.Magic(magic.MAGIC_MIME)
-                    return m.from_file(path)
-                except AttributeError:
-                    pass
-
-                try:    # we try pip filemagic 1.6
-                    m = magic.Magic(flags=magic.MAGIC_MIME_TYPE)
-                    return m.id_filename(path)
-                except AttributeError:
-                    pass
-
-                return "binary/octet-stream"
-            except ImportError:
-                return "binary/octet-stream"
-        raise Exception("FIXME: artifact type not implemented for build environment type %s" % self.get_betype_display())
-
-
     def get_artifact(self, path):
         if self.betype == BuildEnvironment.TYPE_LOCAL:
             return open(path, "r")
bitbake/lib/toaster/toastergui/views.py +1 -1
index 8689a12..2ec0197 100755
@@ -2797,7 +2797,7 @@ if True:
             if file_name is None:
                 raise Exception("Could not handle artifact %s id %s" % (artifact_type, artifact_id))
             else:
-                content_type = b.buildrequest.environment.get_artifact_type(file_name)
+                content_type = MimeTypeFinder.get_mimetype(file_name)
                 fsock = b.buildrequest.environment.get_artifact(file_name)
                 file_name = os.path.basename(file_name) # we assume that the build environment system has the same path conventions as host
 
Signature · 14 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.
Acked-by on v1 1/1
#1 lore raw

Hi Elliot,

The patch looks great! Thanks you!

Quoted 97 lines · Elliot Smith

Artifact download links were broken because the function to get the mimetype for the artifact was incorrectly using the underlying mimetype library. The function was also attached to the build environment controller, which was unnecessary, as we only support local controllers anyway.

Remove the mimetype getter on the build environment and use the one in the view code instead. This works correctly and prevents the download error from occurring.

[YOCTO #8369]

---
 bitbake/lib/toaster/bldcontrol/models.py | 34 --------------------------------
 bitbake/lib/toaster/toastergui/views.py  |  2 +-
 2 files changed, 1 insertion(+), 35 deletions(-)
bitbake/lib/toaster/bldcontrol/models.py +0 -34
index b61de58..c577b10 100644
@@ -39,40 +39,6 @@ class BuildEnvironment(models.Model):
     created     = models.DateTimeField(auto_now_add = True)
     updated     = models.DateTimeField(auto_now = True)
 
-
-    def get_artifact_type(self, path):
-        if self.betype == BuildEnvironment.TYPE_LOCAL:
-            try:
-                import magic
-
-                # fair warning: this is a mess; there are multiple competeing and incompatible
-                # magic modules floating around, so we try some of the most common combinations
-
-                try:    # we try ubuntu's python-magic 5.4
-                    m = magic.open(magic.MAGIC_MIME_TYPE)
-                    m.load()
-                    return m.file(path)
-                except AttributeError:
-                    pass
-
-                try:    # we try python-magic 0.4.6
-                    m = magic.Magic(magic.MAGIC_MIME)
-                    return m.from_file(path)
-                except AttributeError:
-                    pass
-
-                try:    # we try pip filemagic 1.6
-                    m = magic.Magic(flags=magic.MAGIC_MIME_TYPE)
-                    return m.id_filename(path)
-                except AttributeError:
-                    pass
-
-                return "binary/octet-stream"
-            except ImportError:
-                return "binary/octet-stream"
-        raise Exception("FIXME: artifact type not implemented for build environment type %s" % self.get_betype_display())
-
-
     def get_artifact(self, path):
         if self.betype == BuildEnvironment.TYPE_LOCAL:
             return open(path, "r")
bitbake/lib/toaster/toastergui/views.py +1 -1
index 8689a12..2ec0197 100755
@@ -2797,7 +2797,7 @@ if True:
             if file_name is None:
                 raise Exception("Could not handle artifact %s id %s" % (artifact_type, artifact_id))
             else:
-                content_type = b.buildrequest.environment.get_artifact_type(file_name)
+                content_type = MimeTypeFinder.get_mimetype(file_name)
                 fsock = b.buildrequest.environment.get_artifact(file_name)
                 file_name = os.path.basename(file_name) # we assume that the build environment system has the same path conventions as host
 
Signature · 20 lines
Elliot Smith
Software Engineer
Intel OTC

---------------------------------------------------------------------
Intel Corporation (UK) Limited
Registered No. 1134945 (England)
Registered Office: Pipers Way, Swindon SN3 1RJ
VAT No: 860 2173 47

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

-- 
_______________________________________________
toaster mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/toaster
Signature · 3 lines
--
Regards,
Ed