Unicode support for Report Error and Qemu
cover Unicode support for Report Error and Qemu
From: Mariano Lopez <[email protected]>
These two patches add Unicode support for the report error bbclass and the qemu bootlog.
[YOCTO #8225]
The following changes since commit eac61f37e36099f74485dab398b57f3812826d17:
build-appliance-image: Update to jethro head revision (2015-10-01 17:55:11 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib mariano/bug8225p2v2
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/bug8225p2v2
Mariano Lopez (2):
report-error.bbclass: Support Unicode reports oeqa/utils/qemurunner: Add support for Unicode from qemu
meta/classes/report-error.bbclass | 11 +++++++---- meta/lib/oeqa/utils/qemurunner.py | 15 +++++++++++---- 2 files changed, 18 insertions(+), 8 deletions(-)
Signature · 1 line
1.8.4.5
PATCH 1/2 report-error.bbclass: Support Unicode reports
From: Mariano Lopez <[email protected]>
Currently error-report doesn't manage Unicode because the files are opened with the default codec.
This patch changes the codec of the files to UTF-8, this way the reports will include Unicode characters. This is useful for the qemu output when doing the testimage task.
[YOCTO #8225]
--- meta/classes/report-error.bbclass | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-)
meta/classes/report-error.bbclass +7 -4
@@ -9,22 +9,25 @@ ERR_REPORT_DIR ?= "${LOG_DIR}/error-report" def errorreport_getdata(e): + import codecs logpath = e.data.getVar('ERR_REPORT_DIR', True) datafile = os.path.join(logpath, "error-report.txt") - with open(datafile) as f: + with codecs.open(datafile, 'r', 'utf-8') as f: data = f.read() return data def errorreport_savedata(e, newdata, file): import json + import codecs logpath = e.data.getVar('ERR_REPORT_DIR', True) datafile = os.path.join(logpath, file) - with open(datafile, "w") as f: + with codecs.open(datafile, 'w', 'utf-8') as f: json.dump(newdata, f, indent=4, sort_keys=True) return datafile python errorreport_handler () { import json + import codecs logpath = e.data.getVar('ERR_REPORT_DIR', True) datafile = os.path.join(logpath, "error-report.txt")
@@ -53,8 +56,8 @@ python errorreport_handler () { taskdata['task'] = task if log: try: - logFile = open(log, 'r') - logdata = logFile.read().decode('utf-8') + logFile = codecs.open(log, 'r', 'utf-8') + logdata = logFile.read() logFile.close() except: logdata = "Unable to read log file"
Signature · 1 line
1.8.4.5
PATCH 2/2 oeqa/utils/qemurunner: Add support for Unicode from qemu
In reply to [email protected]From: Mariano Lopez <[email protected]>
The current state of qemurunner will drop the Unicode characters received from qemu, this is because error report web had problems with Unicode characters; now that the server support Unicode, it is possible to log all the output from qemu. So far the only Unicode character seen is the copyright symbol.
This patch allows to get Unicode characters from the qemu target and save the log in an UTF-8 file for latter use.
[YOCTO #8225]
--- meta/lib/oeqa/utils/qemurunner.py | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-)
meta/lib/oeqa/utils/qemurunner.py +11 -4
@@ -15,11 +15,18 @@ import select import errno import string import threading +import codecs from oeqa.utils.dump import HostDumper import logging logger = logging.getLogger("BitBake.QemuRunner") +# Get Unicode non printable control chars +control_range = range(0,32)+range(127,160) +control_chars = [unichr(x) for x in control_range + if unichr(x) not in string.printable] +re_control_char = re.compile('[%s]' % re.escape("".join(control_chars))) + class QemuRunner: def __init__(self, machine, rootfs, display, tmpdir, deploy_dir_image, logfile, boottime, dump_dir, dump_host_cmds):
@@ -63,9 +70,9 @@ class QemuRunner: def log(self, msg): if self.logfile: # It is needed to sanitize the data received from qemu - # because is possible to have control characters or Unicode - msg = "".join(filter(lambda x:x in string.printable, msg)) - with open(self.logfile, "a") as f: + # because is possible to have control characters + msg = re_control_char.sub('', unicode(msg, 'utf-8')) + with codecs.open(self.logfile, "a", encoding="utf-8") as f: f.write("%s" % msg) def getOutput(self, o):
@@ -176,7 +183,7 @@ class QemuRunner: cmdline = p.read() # It is needed to sanitize the data received # because is possible to have control characters - cmdline = "".join(filter(lambda x:x in string.printable, cmdline)) + cmdline = re_control_char.sub('', cmdline) try: ips = re.findall("((?:[0-9]{1,3}\.){3}[0-9]{1,3})", cmdline.split("ip=")[1]) if not ips or len(ips) != 3:
Signature · 1 line
1.8.4.5