rationalise mime type guessing in toaster
cover rationalise mime type guessing in toaster
This eliminates the filemagic external requirement and fixes the image download links.
-b
The following changes since commit 19e5f3378e7f6dad9696ef97c074c738dbcef157:
bitbake: toaster: display warnings for bad "IMAGE_FSTYPES" values (2015-10-06 19:50:29 -0700)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib bavery/submit/elliot/2015-09-24_rationalise-mime-type-8369_t2
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=bavery/submit/elliot/2015-09-24_rationalise-mime-type-8369_t2
Elliot Smith (2):
toaster: Use Python's mimetypes module
toaster: Rationalise mimetype guessing to fix artifact downloads
lib/toaster/bldcontrol/models.py | 34 ---------------------------------- lib/toaster/toastergui/views.py | 17 ++++++++++++----- toaster-requirements.txt | 1 - 3 files changed, 12 insertions(+), 40 deletions(-)
--
1.9.1
PATCH 1/2 toaster: Use Python's mimetypes module
From: Elliot Smith <[email protected]>
filemagic is used to guess the mimetype of files when a user requests a download. However, this adds a dependency on an external library.
Python does have a mimetypes module, though this guesses the mimetype rather than doing anything clever with the actual file content. But for our purposes, it's more than adequate. (NB Django also uses this module when serving static files.)
Use this instead of relying on any external code, and remove the filemagic dependency.
--- lib/toaster/toastergui/views.py | 15 +++++++++++---- toaster-requirements.txt | 1 - 2 files changed, 11 insertions(+), 5 deletions(-)
lib/toaster/toastergui/views.py +11 -4
@@ -47,19 +47,26 @@ import json from os.path import dirname from functools import wraps import itertools +import mimetypes -import magic import logging logger = logging.getLogger("toaster") class MimeTypeFinder(object): - _magic = magic.Magic(flags = magic.MAGIC_MIME_TYPE) + # setting this to False enables additional non-standard mimetypes + # to be included in the guess + _strict = False - # returns the mimetype for a file path + # returns the mimetype for a file path as a string, + # or 'application/octet-stream' if the type couldn't be guessed @classmethod def get_mimetype(self, path): - return self._magic.id_filename(path) + guess = mimetypes.guess_type(path, self._strict) + guessed_type = guess[0] + if guessed_type == None: + guessed_type = 'application/octet-stream' + return guessed_type # all new sessions should come through the landing page; # determine in which mode we are running in, and redirect appropriately
toaster-requirements.txt +0 -1
@@ -2,5 +2,4 @@ Django==1.6 South==0.8.4 argparse==1.2.1 wsgiref==0.1.2 -filemagic==1.6 beautifulsoup4>=4.4.0
Signature · 1 line
1.9.1
PATCH 2/2 toaster: Rationalise mimetype guessing to fix artifact downloads
From: Elliot Smith <[email protected]>
Artifact download links were broken because the function to get the mimetype for the artifact was incorrectly using the underlying mimetype library. The function was also attached to the build environment controller, which was unnecessary, as we only support local controllers anyway.
Remove the mimetype getter on the build environment and use the one in the view code instead. This works correctly and prevents the download error from occurring.
[YOCTO #8369]
--- lib/toaster/bldcontrol/models.py | 34 ---------------------------------- lib/toaster/toastergui/views.py | 2 +- 2 files changed, 1 insertion(+), 35 deletions(-)
lib/toaster/bldcontrol/models.py +0 -34
@@ -39,40 +39,6 @@ class BuildEnvironment(models.Model): created = models.DateTimeField(auto_now_add = True) updated = models.DateTimeField(auto_now = True) - - def get_artifact_type(self, path): - if self.betype == BuildEnvironment.TYPE_LOCAL: - try: - import magic - - # fair warning: this is a mess; there are multiple competeing and incompatible - # magic modules floating around, so we try some of the most common combinations - - try: # we try ubuntu's python-magic 5.4 - m = magic.open(magic.MAGIC_MIME_TYPE) - m.load() - return m.file(path) - except AttributeError: - pass - - try: # we try python-magic 0.4.6 - m = magic.Magic(magic.MAGIC_MIME) - return m.from_file(path) - except AttributeError: - pass - - try: # we try pip filemagic 1.6 - m = magic.Magic(flags=magic.MAGIC_MIME_TYPE) - return m.id_filename(path) - except AttributeError: - pass - - return "binary/octet-stream" - except ImportError: - return "binary/octet-stream" - raise Exception("FIXME: artifact type not implemented for build environment type %s" % self.get_betype_display()) - - def get_artifact(self, path): if self.betype == BuildEnvironment.TYPE_LOCAL: return open(path, "r")
lib/toaster/toastergui/views.py +1 -1
@@ -2981,7 +2981,7 @@ if True: if file_name is None: raise Exception("Could not handle artifact %s id %s" % (artifact_type, artifact_id)) else: - content_type = b.buildrequest.environment.get_artifact_type(file_name) + content_type = MimeTypeFinder.get_mimetype(file_name) fsock = b.buildrequest.environment.get_artifact(file_name) file_name = os.path.basename(file_name) # we assume that the build environment system has the same path conventions as host
Signature · 1 line
1.9.1