Skip to contentSkip to conversation

toaster: eventreplay tests

PATCH In review Ed Bartosh 3 participants 7 messages Oct 31, 2016 – Jun 21, 2017

lore
Share & tools
lore

Ed Bartosh

author
#0 lore raw

cover toaster: eventreplay tests

Hi,

This patchset adds 2 tests that use eventreplay script to "replay" core-image-minimal and zlib build events.

Note that tests require event files that are not included into the patchset due to their big size. Please, have a look at README file for the instructions how to produce event files.

The following changes since commit 7c3a5891cb4f8c07bb3092dd6270459816ee9a14:

toaster: tests builds test_core_image_min Clean ups (2016-10-31 11:29:15 +0200)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib ed/toaster/tests/eventreplay
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/tests/eventreplay

Ed Bartosh (4):
toaster: use current directory if BUILDDIR is not set
toaster: add eventreplay test case for core-image-minimal
toaster: add eventreplay test case for zlib
toaster: add tests/eventreplay/README

 bitbake/lib/toaster/orm/models.py                 |  2 +-
 bitbake/lib/toaster/tests/eventreplay/README      | 22 +++++
 bitbake/lib/toaster/tests/eventreplay/__init__.py | 97 +++++++++++++++++++++++
 3 files changed, 120 insertions(+), 1 deletion(-)
 create mode 100644 bitbake/lib/toaster/tests/eventreplay/README
 create mode 100644 bitbake/lib/toaster/tests/eventreplay/__init__.py

--
Regards,
Ed

author
#1 lore raw

PATCH 1/4 toaster: use current directory if BUILDDIR is not set

If BUILDDIR environment variable is not set signal_runbuilds function throws TypeError as os.getenv('BUILDDIR') returns None:

ERROR: unsupported operand type(s) for +=: 'NoneType' and 'str' Traceback (most recent call last):
File "bitbake/lib/bb/ui/toasterui.py", line
391, in main
buildinfohelper.update_build_information(event, errors, warnings,
taskfailures)
File "bitbake/lib/bb/ui/buildinfohelper.py",
line 1184, in update_build_information
self.internal_state['build'], errors, warnings, taskfailures)
File "bitbake/lib/bb/ui/buildinfohelper.py",
line 238, in update_build_stats_and_outcome
signal_runbuilds()
File "bitbake/lib/toaster/orm/models.py", line
1746, in signal_runbuilds

    '.runbuilds.pid')) as pidf:
  File "/usr/lib64/python3.4/posixpath.py", line 82, in join
    path += b

TypeError: unsupported operand type(s) for +=: 'NoneType' and 'str'

Used os.getenv('BUILDIR', '.') to make it always return meaningful directory path. Current directory '.' will be used if BUILDDIR is not set.

---
 bitbake/lib/toaster/orm/models.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
bitbake/lib/toaster/orm/models.py +1 -1
index 9a4db1e..4f8510c 100644
@@ -1742,7 +1742,7 @@ def invalidate_cache(**kwargs):
 def signal_runbuilds():
     """Send SIGUSR1 to runbuilds process"""
     try:
-        with open(os.path.join(os.getenv('BUILDDIR'),
+        with open(os.path.join(os.getenv('BUILDDIR', '.'),
                                '.runbuilds.pid')) as pidf:
             os.kill(int(pidf.read()), SIGUSR1)
     except FileNotFoundError:
Signature · 1 line
2.1.4
author
#2 lore raw

PATCH 2/4 toaster: add eventreplay test case for core-image-minimal

Run toaster-eventreplay with core-image-minimal.events and test if all required packages present in Target_Installed_Package table.

---
 bitbake/lib/toaster/tests/eventreplay/__init__.py | 87 +++++++++++++++++++++++
 1 file changed, 87 insertions(+)
 create mode 100644 bitbake/lib/toaster/tests/eventreplay/__init__.py
bitbake/lib/toaster/tests/eventreplay/__init__.py new file +87 -0
new file mode 100644
index 0000000..792f07d
@@ -0,0 +1,87 @@
+#! /usr/bin/env python
+# ex:ts=4:sw=4:sts=4:et
+# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
+#
+# BitBake Toaster Implementation
+#
+# Copyright (C) 2016 Intel Corporation
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+
+# Tests were part of openembedded-core oe selftest Authored by: Lucian Musat
+# Ionut Chisanovici, Paul Eggleton and Cristian Iorga
+
+"""
+Test toaster backend by playing build event log files
+using toaster-eventreplay script
+"""
+
+import os
+
+from subprocess import getstatusoutput
+from pathlib import Path
+
+from django.test import TestCase
+
+from orm.models import Target_Installed_Package
+
+class EventReplay(TestCase):
+    """Base class for eventreplay test cases"""
+
+    def setUp(self):
+        """
+        Setup build environment:
+            - set self.script to toaster-eventreplay path
+            - set self.eventplay_dir to the value of EVENTPLAY_DIR env variable
+        """
+        bitbake_dir = Path(__file__.split('lib/toaster')[0])
+        self.script = bitbake_dir /  'bin' / 'toaster-eventreplay'
+        self.assertTrue(self.script.exists(), "%s doesn't exist")
+        self.eventplay_dir = os.getenv("EVENTREPLAY_DIR")
+        self.assertTrue(self.eventplay_dir,
+                        "Environment variable EVENTREPLAY_DIR is not set")
+
+    def _replay(self, eventfile):
+        """Run toaster-eventplay <eventfile>"""
+        eventpath = Path(self.eventplay_dir) / eventfile
+        status, output = getstatusoutput('%s %s' % (self.script, eventpath))
+        if status:
+            print(output)
+
+        self.assertEqual(status, 0)
+
+class CoreImageMinimalEventReplay(EventReplay):
+    """Replay core-image-minimal events"""
+
+    def test_installed_packages(self):
+        """Test if all required packages have been installed"""
+
+        self._replay('core-image-minimal.events')
+
+        # test installed packages
+        packages = sorted(Target_Installed_Package.objects.\
+                          values_list('package__name', flat=True))
+        self.assertEqual(packages, ['base-files', 'base-passwd', 'busybox',
+                                    'busybox-hwclock', 'busybox-syslog',
+                                    'busybox-udhcpc', 'eudev', 'glibc',
+                                    'init-ifupdown', 'initscripts',
+                                    'initscripts-functions', 'kernel-base',
+                                    'kernel-module-uvesafb', 'libkmod',
+                                    'modutils-initscripts', 'netbase',
+                                    'packagegroup-core-boot', 'run-postinsts',
+                                    'sysvinit', 'sysvinit-inittab',
+                                    'sysvinit-pidof', 'udev-cache',
+                                    'update-alternatives-opkg',
+                                    'update-rc.d', 'util-linux-libblkid',
+                                    'util-linux-libuuid', 'v86d', 'zlib'])
Signature · 1 line
2.1.4
author
#3 lore raw

PATCH 3/4 toaster: add eventreplay test case for zlib

In reply to Ed Bartosh

Run toaster-eventreplay with zlib.events.
Check if zlib build and package present in Toaster database.

---
 bitbake/lib/toaster/tests/eventreplay/__init__.py | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
bitbake/lib/toaster/tests/eventreplay/__init__.py +11 -1
index 792f07d..6956619 100644
@@ -34,7 +34,7 @@ from pathlib import Path
 
 from django.test import TestCase
 
-from orm.models import Target_Installed_Package
+from orm.models import Target_Installed_Package, Package, Build
 
 class EventReplay(TestCase):
     """Base class for eventreplay test cases"""
@@ -85,3 +85,13 @@ class CoreImageMinimalEventReplay(EventReplay):
                                     'update-alternatives-opkg',
                                     'update-rc.d', 'util-linux-libblkid',
                                     'util-linux-libuuid', 'v86d', 'zlib'])
+
+class ZlibEventReplay(EventReplay):
+    """Replay zlib events"""
+
+    def test_replay_zlib(self):
+        """Test if zlib build and package are in the database"""
+        self._replay("zlib.events")
+
+        self.assertEqual(Build.objects.last().target_set.last().target, "zlib")
+        self.assertTrue('zlib' in Package.objects.values_list('name', flat=True))
Signature · 1 line
2.1.4
author
#4 lore raw

PATCH 4/4 toaster: add tests/eventreplay/README

In reply to Ed Bartosh

Put instructions on how to prepare event log files and run eventreplay tests.

---
 bitbake/lib/toaster/tests/eventreplay/README | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
 create mode 100644 bitbake/lib/toaster/tests/eventreplay/README
bitbake/lib/toaster/tests/eventreplay/README new file +22 -0
new file mode 100644
index 0000000..8c5bb64
@@ -0,0 +1,22 @@
+# Running eventreplay tests
+
+These tests use event log files produced by bitbake <target> -w <event log file>
+You need to have event log files produced before running this tests.
+
+At the moment of writing this document tests use 2 event log files: zlib.events
+and core-image-minimal.events. They're not provided with the tests due to their
+significant size.
+
+Here is how to produce them:
+
+$ . oe-init-build-env
+$ rm -r tmp sstate-cache
+$ bitbake core-image-minimal -w core-image-minimal.events
+$ rm -rf tmp sstate-cache
+$ bitbake zlib -w zlib.events
+
+After that it should be possible to run eventreplay tests this way:
+
+$ EVENTREPLAY_DIR=./ DJANGO_SETTINGS_MODULE=toastermain.settings_test ../bitbake/lib/toaster/manage.py test -v2 tests.eventreplay
+
+Note that environment variable EVENTREPLAY_DIR should point to the directory with event log files.
Signature · 1 line
2.1.4
#5 lore raw

Re: [PATCH 4/4] toaster: add tests/eventreplay/README

Thanks. Applied to toaster-next and sent bitbake-devel

Quoted 37 lines · Ed Bartosh

Put instructions on how to prepare event log files and run eventreplay tests.

---
  bitbake/lib/toaster/tests/eventreplay/README | 22 ++++++++++++++++++++++
  1 file changed, 22 insertions(+)

create mode 100644 bitbake/lib/toaster/tests/eventreplay/README

bitbake/lib/toaster/tests/eventreplay/README new file +22 -0
new file mode 100644
index 0000000..8c5bb64
@@ -0,0 +1,22 @@
+# Running eventreplay tests
+
+These tests use event log files produced by bitbake <target> -w <event log file>
+You need to have event log files produced before running this tests.
+
+At the moment of writing this document tests use 2 event log files: zlib.events
+and core-image-minimal.events. They're not provided with the tests due to their
+significant size.
+
+Here is how to produce them:
+
+$ . oe-init-build-env
+$ rm -r tmp sstate-cache
+$ bitbake core-image-minimal -w core-image-minimal.events
+$ rm -rf tmp sstate-cache
+$ bitbake zlib -w zlib.events
+
+After that it should be possible to run eventreplay tests this way:
+
+$ EVENTREPLAY_DIR=./ DJANGO_SETTINGS_MODULE=toastermain.settings_test ../bitbake/lib/toaster/manage.py test -v2 tests.eventreplay
+
+Note that environment variable EVENTREPLAY_DIR should point to the directory with event log files.
#6 lore raw

cover toaster: eventreplay tests

In reply to Ed Bartosh

Hi,

There was interest in knowing how to replay bitbake events through a toaster instance in an offline mode. I'm reforwarding the test eventreplay commit to the list as it has the relevant information. The README (
http://git.yoctoproject.org/cgit.cgi/poky-contrib/commit/?h=ed/toaster/tests/eventreplay&id=92dd8d7561657c769d9d3b460cdce6f1f638547f)
has the instructions.

-bavery
an intel employee

---------- Forwarded message ----------
From: Ed Bartosh <[email protected]>
Date: Mon, Oct 31, 2016 at 3:16 AM
Subject: [Toaster] [PATCH 0/4] toaster: eventreplay tests
To: [email protected]

Hi,

This patchset adds 2 tests that use eventreplay script to "replay" core-image-minimal
and zlib build events.

Note that tests require event files that are not included into the patchset due to
their big size. Please, have a look at README file for the instructions how to produce
event files.

The following changes since commit 7c3a5891cb4f8c07bb3092dd6270459816ee9a14:

toaster: tests builds test_core_image_min Clean ups (2016-10-31 11:29:15
+0200)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib ed/toaster/tests/eventreplay
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=
ed/toaster/tests/eventreplay

Ed Bartosh (4):
toaster: use current directory if BUILDDIR is not set
toaster: add eventreplay test case for core-image-minimal
toaster: add eventreplay test case for zlib
toaster: add tests/eventreplay/README

 bitbake/lib/toaster/orm/models.py                 |  2 +-
 bitbake/lib/toaster/tests/eventreplay/README      | 22 +++++
 bitbake/lib/toaster/tests/eventreplay/__init__.py | 97
+++++++++++++++++++++++
 3 files changed, 120 insertions(+), 1 deletion(-)
 create mode 100644 bitbake/lib/toaster/tests/eventreplay/README
 create mode 100644 bitbake/lib/toaster/tests/eventreplay/__init__.py

--
Regards,
Ed

--
_______________________________________________
toaster mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/toaster