toaster: eventreplay tests
cover toaster: eventreplay tests
Hi,
This patchset adds 2 tests that use eventreplay script to "replay" core-image-minimal and zlib build events.
Note that tests require event files that are not included into the patchset due to their big size. Please, have a look at README file for the instructions how to produce event files.
The following changes since commit 7c3a5891cb4f8c07bb3092dd6270459816ee9a14:
toaster: tests builds test_core_image_min Clean ups (2016-10-31 11:29:15 +0200)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib ed/toaster/tests/eventreplay
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/tests/eventreplay
Ed Bartosh (4):
toaster: use current directory if BUILDDIR is not set
toaster: add eventreplay test case for core-image-minimal
toaster: add eventreplay test case for zlib
toaster: add tests/eventreplay/README
bitbake/lib/toaster/orm/models.py | 2 +- bitbake/lib/toaster/tests/eventreplay/README | 22 +++++ bitbake/lib/toaster/tests/eventreplay/__init__.py | 97 +++++++++++++++++++++++ 3 files changed, 120 insertions(+), 1 deletion(-) create mode 100644 bitbake/lib/toaster/tests/eventreplay/README create mode 100644 bitbake/lib/toaster/tests/eventreplay/__init__.py
--
Regards,
Ed
PATCH 1/4 toaster: use current directory if BUILDDIR is not set
If BUILDDIR environment variable is not set signal_runbuilds function throws TypeError as os.getenv('BUILDDIR') returns None:
ERROR: unsupported operand type(s) for +=: 'NoneType' and 'str' Traceback (most recent call last):
File "bitbake/lib/bb/ui/toasterui.py", line
391, in main
buildinfohelper.update_build_information(event, errors, warnings,
taskfailures)
File "bitbake/lib/bb/ui/buildinfohelper.py",
line 1184, in update_build_information
self.internal_state['build'], errors, warnings, taskfailures)
File "bitbake/lib/bb/ui/buildinfohelper.py",
line 238, in update_build_stats_and_outcome
signal_runbuilds()
File "bitbake/lib/toaster/orm/models.py", line
1746, in signal_runbuilds
'.runbuilds.pid')) as pidf:
File "/usr/lib64/python3.4/posixpath.py", line 82, in join
path += bTypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'
Used os.getenv('BUILDIR', '.') to make it always return meaningful directory path. Current directory '.' will be used if BUILDDIR is not set.
--- bitbake/lib/toaster/orm/models.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
bitbake/lib/toaster/orm/models.py +1 -1
@@ -1742,7 +1742,7 @@ def invalidate_cache(**kwargs): def signal_runbuilds(): """Send SIGUSR1 to runbuilds process""" try: - with open(os.path.join(os.getenv('BUILDDIR'), + with open(os.path.join(os.getenv('BUILDDIR', '.'), '.runbuilds.pid')) as pidf: os.kill(int(pidf.read()), SIGUSR1) except FileNotFoundError:
Signature · 1 line
2.1.4
PATCH 2/4 toaster: add eventreplay test case for core-image-minimal
Run toaster-eventreplay with core-image-minimal.events and test if all required packages present in Target_Installed_Package table.
--- bitbake/lib/toaster/tests/eventreplay/__init__.py | 87 +++++++++++++++++++++++ 1 file changed, 87 insertions(+) create mode 100644 bitbake/lib/toaster/tests/eventreplay/__init__.py
bitbake/lib/toaster/tests/eventreplay/__init__.py new file +87 -0
@@ -0,0 +1,87 @@ +#! /usr/bin/env python +# ex:ts=4:sw=4:sts=4:et +# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*- +# +# BitBake Toaster Implementation +# +# Copyright (C) 2016 Intel Corporation +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License version 2 as +# published by the Free Software Foundation. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License along +# with this program; if not, write to the Free Software Foundation, Inc., +# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + +# Tests were part of openembedded-core oe selftest Authored by: Lucian Musat +# Ionut Chisanovici, Paul Eggleton and Cristian Iorga + +""" +Test toaster backend by playing build event log files +using toaster-eventreplay script +""" + +import os + +from subprocess import getstatusoutput +from pathlib import Path + +from django.test import TestCase + +from orm.models import Target_Installed_Package + +class EventReplay(TestCase): + """Base class for eventreplay test cases""" + + def setUp(self): + """ + Setup build environment: + - set self.script to toaster-eventreplay path + - set self.eventplay_dir to the value of EVENTPLAY_DIR env variable + """ + bitbake_dir = Path(__file__.split('lib/toaster')[0]) + self.script = bitbake_dir / 'bin' / 'toaster-eventreplay' + self.assertTrue(self.script.exists(), "%s doesn't exist") + self.eventplay_dir = os.getenv("EVENTREPLAY_DIR") + self.assertTrue(self.eventplay_dir, + "Environment variable EVENTREPLAY_DIR is not set") + + def _replay(self, eventfile): + """Run toaster-eventplay <eventfile>""" + eventpath = Path(self.eventplay_dir) / eventfile + status, output = getstatusoutput('%s %s' % (self.script, eventpath)) + if status: + print(output) + + self.assertEqual(status, 0) + +class CoreImageMinimalEventReplay(EventReplay): + """Replay core-image-minimal events""" + + def test_installed_packages(self): + """Test if all required packages have been installed""" + + self._replay('core-image-minimal.events') + + # test installed packages + packages = sorted(Target_Installed_Package.objects.\ + values_list('package__name', flat=True)) + self.assertEqual(packages, ['base-files', 'base-passwd', 'busybox', + 'busybox-hwclock', 'busybox-syslog', + 'busybox-udhcpc', 'eudev', 'glibc', + 'init-ifupdown', 'initscripts', + 'initscripts-functions', 'kernel-base', + 'kernel-module-uvesafb', 'libkmod', + 'modutils-initscripts', 'netbase', + 'packagegroup-core-boot', 'run-postinsts', + 'sysvinit', 'sysvinit-inittab', + 'sysvinit-pidof', 'udev-cache', + 'update-alternatives-opkg', + 'update-rc.d', 'util-linux-libblkid', + 'util-linux-libuuid', 'v86d', 'zlib'])
Signature · 1 line
2.1.4
PATCH 3/4 toaster: add eventreplay test case for zlib
In reply to Ed BartoshRun toaster-eventreplay with zlib.events.
Check if zlib build and package present in Toaster database.
--- bitbake/lib/toaster/tests/eventreplay/__init__.py | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
bitbake/lib/toaster/tests/eventreplay/__init__.py +11 -1
@@ -34,7 +34,7 @@ from pathlib import Path from django.test import TestCase -from orm.models import Target_Installed_Package +from orm.models import Target_Installed_Package, Package, Build class EventReplay(TestCase): """Base class for eventreplay test cases"""
@@ -85,3 +85,13 @@ class CoreImageMinimalEventReplay(EventReplay): 'update-alternatives-opkg', 'update-rc.d', 'util-linux-libblkid', 'util-linux-libuuid', 'v86d', 'zlib']) + +class ZlibEventReplay(EventReplay): + """Replay zlib events""" + + def test_replay_zlib(self): + """Test if zlib build and package are in the database""" + self._replay("zlib.events") + + self.assertEqual(Build.objects.last().target_set.last().target, "zlib") + self.assertTrue('zlib' in Package.objects.values_list('name', flat=True))
Signature · 1 line
2.1.4
PATCH 4/4 toaster: add tests/eventreplay/README
In reply to Ed BartoshPut instructions on how to prepare event log files and run eventreplay tests.
--- bitbake/lib/toaster/tests/eventreplay/README | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) create mode 100644 bitbake/lib/toaster/tests/eventreplay/README
bitbake/lib/toaster/tests/eventreplay/README new file +22 -0
@@ -0,0 +1,22 @@ +# Running eventreplay tests + +These tests use event log files produced by bitbake <target> -w <event log file> +You need to have event log files produced before running this tests. + +At the moment of writing this document tests use 2 event log files: zlib.events +and core-image-minimal.events. They're not provided with the tests due to their +significant size. + +Here is how to produce them: + +$ . oe-init-build-env +$ rm -r tmp sstate-cache +$ bitbake core-image-minimal -w core-image-minimal.events +$ rm -rf tmp sstate-cache +$ bitbake zlib -w zlib.events + +After that it should be possible to run eventreplay tests this way: + +$ EVENTREPLAY_DIR=./ DJANGO_SETTINGS_MODULE=toastermain.settings_test ../bitbake/lib/toaster/manage.py test -v2 tests.eventreplay + +Note that environment variable EVENTREPLAY_DIR should point to the directory with event log files.
Signature · 1 line
2.1.4
Re: [PATCH 4/4] toaster: add tests/eventreplay/README
Thanks. Applied to toaster-next and sent bitbake-devel
Quoted 37 lines · Ed Bartosh
Put instructions on how to prepare event log files and run eventreplay tests.
Signed-off-by Ed Bartosh <[email protected]>--- bitbake/lib/toaster/tests/eventreplay/README | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+)create mode 100644 bitbake/lib/toaster/tests/eventreplay/README
bitbake/lib/toaster/tests/eventreplay/README new file +22
-0@@ -0,0 +1,22 @@ +# Running eventreplay tests + +These tests use event log files produced by bitbake <target> -w <event log file> +You need to have event log files produced before running this tests. + +At the moment of writing this document tests use 2 event log files: zlib.events +and core-image-minimal.events. They're not provided with the tests due to their +significant size. + +Here is how to produce them: + +$ . oe-init-build-env +$ rm -r tmp sstate-cache +$ bitbake core-image-minimal -w core-image-minimal.events +$ rm -rf tmp sstate-cache +$ bitbake zlib -w zlib.events + +After that it should be possible to run eventreplay tests this way: + +$ EVENTREPLAY_DIR=./ DJANGO_SETTINGS_MODULE=toastermain.settings_test ../bitbake/lib/toaster/manage.py test -v2 tests.eventreplay + +Note that environment variable EVENTREPLAY_DIR should point to the directory with event log files.
cover toaster: eventreplay tests
In reply to Ed BartoshHi,
There was interest in knowing how to replay bitbake events through a toaster instance in an offline mode. I'm reforwarding the test eventreplay commit to the list as it has the relevant information. The README (
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=ed/toaster/tests/eventreplay&id=92dd8d7561657c769d9d3b460cdce6f1f638547f)
has the instructions.
-bavery
an intel employee
---------- Forwarded message ----------
From: Ed Bartosh <[email protected]>
Date: Mon, Oct 31, 2016 at 3:16 AM
Subject: [Toaster] [PATCH 0/4] toaster: eventreplay tests
To: [email protected]
Hi,
This patchset adds 2 tests that use eventreplay script to "replay" core-image-minimal
and zlib build events.
Note that tests require event files that are not included into the patchset due to
their big size. Please, have a look at README file for the instructions how to produce
event files.
The following changes since commit 7c3a5891cb4f8c07bb3092dd6270459816ee9a14:
toaster: tests builds test_core_image_min Clean ups (2016-10-31 11:29:15
+0200)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib ed/toaster/tests/eventreplay
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=
ed/toaster/tests/eventreplay
Ed Bartosh (4):
toaster: use current directory if BUILDDIR is not set
toaster: add eventreplay test case for core-image-minimal
toaster: add eventreplay test case for zlib
toaster: add tests/eventreplay/README
bitbake/lib/toaster/orm/models.py | 2 +- bitbake/lib/toaster/tests/eventreplay/README | 22 +++++ bitbake/lib/toaster/tests/eventreplay/__init__.py | 97 +++++++++++++++++++++++ 3 files changed, 120 insertions(+), 1 deletion(-) create mode 100644 bitbake/lib/toaster/tests/eventreplay/README create mode 100644 bitbake/lib/toaster/tests/eventreplay/__init__.py
--
Regards,
Ed
--
_______________________________________________
toaster mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/toaster