oeqa/runtime: Import custom targets

PATCH In review [email protected] 2 participants 6 messages Mar 24, 2017 – Mar 27, 2017

lore

Share & tools

lore

[email protected]

author

2017-03-24 22:06 UTC #0 lore raw

cover oeqa/runtime: Import custom targets

From: Mariano Lopez <[email protected]>

This series allows to import custom targets to be used with runtime testing. Now is possible to have a target in <layer>/lib/oeqa/core/target and testimage and testexport can use such target using TEST_TARGET variable.

To register a custom target you just need use decorate a target class with registerTarget, and set "targetName" attribute to the name that will be used by TEST_TARGET variable.

The following changes since commit 9fe7a69535f9443175da8289633a761f58c372ff:

bitbake: bitbake-diffsigs: Add debug support (2017-03-23 13:19:50 +0000)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib mariano/oeqa_target
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=mariano/oeqa_target

Mariano Lopez (3):
oeqa: Remove __init__.py files.
oeqa/target: Add decorator to register targets oeqa/runtime/context.py: Add automatic target loading

 meta/classes/testexport.bbclass                    |  19 +++-
 meta/classes/testimage.bbclass                     |   4 +-
 meta/lib/oeqa/core/__init__.py                     |   0
 meta/lib/oeqa/core/cases/__init__.py               |   0
 .../oeqa/core/decorator/{__init__.py => base.py}   |   0
 meta/lib/oeqa/core/decorator/data.py               |   3 +-
 meta/lib/oeqa/core/decorator/depends.py            |   4 +-
 meta/lib/oeqa/core/decorator/oeid.py               |   2 +-
 meta/lib/oeqa/core/decorator/oetag.py              |   2 +-
 meta/lib/oeqa/core/decorator/oetimeout.py          |   2 +-
 meta/lib/oeqa/core/loader.py                       |   2 +-
 meta/lib/oeqa/core/target/__init__.py              |  33 -------
 meta/lib/oeqa/core/target/base.py                  |  82 ++++++++++++++++
 meta/lib/oeqa/core/target/qemu.py                  |   7 +-
 meta/lib/oeqa/core/target/ssh.py                   |   6 +-
 meta/lib/oeqa/core/tests/__init__.py               |   0
 meta/lib/oeqa/core/utils/__init__.py               |   0
 meta/lib/oeqa/runtime/context.py                   | 109 +++++----------------
 meta/lib/oeqa/runtime/decorator/package.py         |   2 +-
 meta/lib/oeqa/runtime/utils/__init__.py            |   0
 meta/lib/oeqa/sdk/__init__.py                      |   0
 meta/lib/oeqa/sdk/utils/__init__.py                |   0
 meta/lib/oeqa/sdkext/__init__.py                   |   0
 23 files changed, 142 insertions(+), 135 deletions(-)
 delete mode 100644 meta/lib/oeqa/core/__init__.py
 delete mode 100644 meta/lib/oeqa/core/cases/__init__.py
 rename meta/lib/oeqa/core/decorator/{__init__.py => base.py} (100%)
 delete mode 100644 meta/lib/oeqa/core/target/__init__.py
 create mode 100644 meta/lib/oeqa/core/target/base.py
 delete mode 100644 meta/lib/oeqa/core/tests/__init__.py
 delete mode 100644 meta/lib/oeqa/core/utils/__init__.py
 delete mode 100644 meta/lib/oeqa/runtime/utils/__init__.py
 delete mode 100644 meta/lib/oeqa/sdk/__init__.py
 delete mode 100644 meta/lib/oeqa/sdk/utils/__init__.py
 delete mode 100644 meta/lib/oeqa/sdkext/__init__.py

Signature · 1 line

2.10.2

[email protected]

author

2017-03-24 22:06 UTC #1 lore raw

PATCH 1/3 oeqa: Remove __init__.py files.

From: Mariano Lopez <[email protected]>

From python 3.3 and above, if a module path contains an __init__.py file, just that path will be checked for module imports; this behavior makes impossible to extend core functionality using other layers with the current __init__.py files. You can check a more detailed explanation on:
http://python-notes.curiousefficiency.org/en/latest/python_concepts/import_traps.html

This patch removes the __init__.py files from just from the new framework.

Signed-off-by Mariano Lopez <[email protected]>

---
 meta/lib/oeqa/core/__init__.py                        | 0
 meta/lib/oeqa/core/cases/__init__.py                  | 0
 meta/lib/oeqa/core/decorator/{__init__.py => base.py} | 0
 meta/lib/oeqa/core/decorator/data.py                  | 3 +--
 meta/lib/oeqa/core/decorator/depends.py               | 4 +---
 meta/lib/oeqa/core/decorator/oeid.py                  | 2 +-
 meta/lib/oeqa/core/decorator/oetag.py                 | 2 +-
 meta/lib/oeqa/core/decorator/oetimeout.py             | 2 +-
 meta/lib/oeqa/core/loader.py                          | 2 +-
 meta/lib/oeqa/core/target/{__init__.py => base.py}    | 0
 meta/lib/oeqa/core/target/qemu.py                     | 2 +-
 meta/lib/oeqa/core/target/ssh.py                      | 2 +-
 meta/lib/oeqa/core/tests/__init__.py                  | 0
 meta/lib/oeqa/core/utils/__init__.py                  | 0
 meta/lib/oeqa/runtime/decorator/package.py            | 2 +-
 meta/lib/oeqa/runtime/utils/__init__.py               | 0
 meta/lib/oeqa/sdk/__init__.py                         | 0
 meta/lib/oeqa/sdk/utils/__init__.py                   | 0
 meta/lib/oeqa/sdkext/__init__.py                      | 0
 19 files changed, 9 insertions(+), 12 deletions(-)
 delete mode 100644 meta/lib/oeqa/core/__init__.py
 delete mode 100644 meta/lib/oeqa/core/cases/__init__.py
 rename meta/lib/oeqa/core/decorator/{__init__.py => base.py} (100%)
 rename meta/lib/oeqa/core/target/{__init__.py => base.py} (100%)
 delete mode 100644 meta/lib/oeqa/core/tests/__init__.py
 delete mode 100644 meta/lib/oeqa/core/utils/__init__.py
 delete mode 100644 meta/lib/oeqa/runtime/utils/__init__.py
 delete mode 100644 meta/lib/oeqa/sdk/__init__.py
 delete mode 100644 meta/lib/oeqa/sdk/utils/__init__.py
 delete mode 100644 meta/lib/oeqa/sdkext/__init__.py

meta/lib/oeqa/core/__init__.py deleted +0 -0

deleted file mode 100644
index e69de29..0000000

meta/lib/oeqa/core/cases/__init__.py deleted +0 -0

deleted file mode 100644
index e69de29..0000000

meta/lib/oeqa/core/decorator/base.py renamed from meta/lib/oeqa/core/decorator/__init__.py +0 -0

similarity index 100%
rename from meta/lib/oeqa/core/decorator/__init__.py
rename to meta/lib/oeqa/core/decorator/base.py

meta/lib/oeqa/core/decorator/data.py +1 -2

index ff7bdd9..658e8a2 100644
@@ -2,8 +2,7 @@
 # Released under the MIT license (see COPYING.MIT)
 
 from oeqa.core.exception import OEQAMissingVariable
-
-from . import OETestDecorator, registerDecorator
+from oeqa.core.decorator.base import OETestDecorator, registerDecorator
 
 def has_feature(td, feature):
     """

meta/lib/oeqa/core/decorator/depends.py +1 -3

index 195711c..608b06a 100644
@@ -2,10 +2,8 @@
 # Released under the MIT license (see COPYING.MIT)
 
 from unittest import SkipTest
-
 from oeqa.core.exception import OEQADependency
-
-from . import OETestDiscover, registerDecorator
+from oeqa.core.decorator.base import OETestDiscover, registerDecorator
 
 def _add_depends(registry, case, depends):
     module_name = case.__module__

meta/lib/oeqa/core/decorator/oeid.py +1 -1

index ea8017a..a5bee52 100644
@@ -1,8 +1,8 @@
 # Copyright (C) 2016 Intel Corporation
 # Released under the MIT license (see COPYING.MIT)
 
-from . import OETestFilter, registerDecorator
 from oeqa.core.utils.misc import intToList
+from oeqa.core.decorator.base import OETestFilter, registerDecorator
 
 def _idFilter(oeid, filters):
      return False if oeid in filters else True

meta/lib/oeqa/core/decorator/oetag.py +1 -1

index ad38ab7..6a36428 100644
@@ -1,8 +1,8 @@
 # Copyright (C) 2016 Intel Corporation
 # Released under the MIT license (see COPYING.MIT)
 
-from . import OETestFilter, registerDecorator
 from oeqa.core.utils.misc import strToList
+from oeqa.core.decorator.base import OETestFilter, registerDecorator
 
 def _tagFilter(tags, filters):
     return False if set(tags) & set(filters) else True

meta/lib/oeqa/core/decorator/oetimeout.py +1 -1

index a247583..749e8a1 100644
@@ -2,8 +2,8 @@
 # Released under the MIT license (see COPYING.MIT)
 
 import signal
-from . import OETestDecorator, registerDecorator
 from oeqa.core.exception import OEQATimeoutError
+from oeqa.core.decorator.base import OETestDecorator, registerDecorator
 
 @registerDecorator
 class OETimeout(OETestDecorator):

meta/lib/oeqa/core/loader.py +1 -1

index 74f1117..2e7d0d0 100644
@@ -9,7 +9,7 @@ from oeqa.core.utils.path import findFile
 from oeqa.core.utils.test import getSuiteModules, getCaseID
 
 from oeqa.core.case import OETestCase
-from oeqa.core.decorator import decoratorClasses, OETestDecorator, \
+from oeqa.core.decorator.base import decoratorClasses, OETestDecorator, \
         OETestFilter, OETestDiscover
 
 def _make_failed_test(classname, methodname, exception, suiteClass):

meta/lib/oeqa/core/target/base.py renamed from meta/lib/oeqa/core/target/__init__.py +0 -0

similarity index 100%
rename from meta/lib/oeqa/core/target/__init__.py
rename to meta/lib/oeqa/core/target/base.py

meta/lib/oeqa/core/target/qemu.py +1 -1

index 2dc521c..261c466 100644
@@ -6,7 +6,7 @@ import sys
 import signal
 import time
 
-from .ssh import OESSHTarget
+from oeqa.core.target.ssh import OESSHTarget
 from oeqa.utils.qemurunner import QemuRunner
 
 supported_fstypes = ['ext3', 'ext4', 'cpio.gz', 'wic', 'elf']

meta/lib/oeqa/core/target/ssh.py +1 -1

index b80939c..035965a 100644
@@ -7,7 +7,7 @@ import select
 import logging
 import subprocess
 
-from . import OETarget
+from oeqa.core.target.base import OETarget
 
 class OESSHTarget(OETarget):
     def __init__(self, logger, ip, server_ip, timeout=300, user='root',

meta/lib/oeqa/core/tests/__init__.py deleted +0 -0

deleted file mode 100644
index e69de29..0000000

meta/lib/oeqa/core/utils/__init__.py deleted +0 -0

deleted file mode 100644
index e69de29..0000000

meta/lib/oeqa/runtime/decorator/package.py +1 -1

index aa6ecb6..0d3224f 100644
@@ -1,8 +1,8 @@
 # Copyright (C) 2016 Intel Corporation
 # Released under the MIT license (see COPYING.MIT)
 
-from oeqa.core.decorator import OETestDecorator, registerDecorator
 from oeqa.core.utils.misc import strToSet
+from oeqa.core.decorator.base import OETestDecorator, registerDecorator
 
 @registerDecorator
 class OEHasPackage(OETestDecorator):

meta/lib/oeqa/runtime/utils/__init__.py deleted +0 -0

deleted file mode 100644
index e69de29..0000000

meta/lib/oeqa/sdk/__init__.py deleted +0 -0

deleted file mode 100644
index e69de29..0000000

meta/lib/oeqa/sdk/utils/__init__.py deleted +0 -0

deleted file mode 100644
index e69de29..0000000

meta/lib/oeqa/sdkext/__init__.py deleted +0 -0

deleted file mode 100644
index e69de29..0000000

Signature · 1 line

2.10.2

[email protected]

author

2017-03-24 22:06 UTC #2 lore raw

PATCH 2/3 oeqa/target: Add decorator to register targets

In reply to [email protected]

From: Mariano Lopez <[email protected]>

This adds a way to register targets that can be used with runtime testing. To do this just decorate a target class with registerTarget, and set "targetName" attribute to the name that will be used by TEST_TARGET variable.

Signed-off-by Mariano Lopez <[email protected]>

---
 meta/lib/oeqa/core/target/base.py | 23 +++++++++++++++++++++++
 meta/lib/oeqa/core/target/qemu.py |  5 +++++
 meta/lib/oeqa/core/target/ssh.py  |  6 +++++-
 3 files changed, 33 insertions(+), 1 deletion(-)

meta/lib/oeqa/core/target/base.py +23 -0

index d2468bc..c886491 100644
@@ -1,8 +1,31 @@
 # Copyright (C) 2016 Intel Corporation
 # Released under the MIT license (see COPYING.MIT)
+import os
+import sys
+import importlib
 
 from abc import abstractmethod
 
+# Used to keep record of registered targets, it
+# uses class' targetName as the key to the class.
+targetClasses = {}
+
+def registerTarget(obj):
+    """ Use as decorator to register targets for runtime testing """
+
+    if (obj.targetName in targetClasses and
+        obj.__name__ != targetClasses[obj.targetName].__name__):
+
+        msg = ('Tried to register %s as "%s" that is used by %s' %
+               (obj, obj.targetName, targetClasses[obj.targetName]))
+        raise ImportError(msg)
+
+    if not issubclass(obj, OETarget):
+        raise TypeError('%s must inherit from OETarget' % obj)
+
+    targetClasses[obj.targetName] = obj
+    return obj
+
 class OETarget(object):
 
     def __init__(self, logger, *args, **kwargs):

meta/lib/oeqa/core/target/qemu.py +5 -0

index 261c466..d1b6bf5 100644
@@ -6,12 +6,17 @@ import sys
 import signal
 import time
 
+from oeqa.core.target.base import registerTarget
 from oeqa.core.target.ssh import OESSHTarget
 from oeqa.utils.qemurunner import QemuRunner
 
 supported_fstypes = ['ext3', 'ext4', 'cpio.gz', 'wic', 'elf']
 
+@registerTarget
 class OEQemuTarget(OESSHTarget):
+
+    targetName = 'qemu'
+
     def __init__(self, logger, ip, server_ip, timeout=300, user='root',
             port=None, machine='', rootfs='', kernel='', kvm=False,
             dump_dir='', dump_host_cmds='', display='', bootlog='',

meta/lib/oeqa/core/target/ssh.py +5 -1

index 035965a..d973058 100644
@@ -7,9 +7,13 @@ import select
 import logging
 import subprocess
 
-from oeqa.core.target.base import OETarget
+from oeqa.core.target.base import OETarget, registerTarget
 
+@registerTarget
 class OESSHTarget(OETarget):
+
+    targetName = 'simpleremote'
+
     def __init__(self, logger, ip, server_ip, timeout=300, user='root',
                  port=None, **kwargs):
         if not logger:

Signature · 1 line

2.10.2

[email protected]

author

2017-03-24 22:06 UTC #3 lore raw

PATCH 3/3 oeqa/runtime/context.py: Add automatic target loading

In reply to [email protected]

From: Mariano Lopez <[email protected]>

This adds automatic target loading for runtime testing. It will look for targets in 'lib/oeqa/core/target' in every layer used in the build.

Introspection was used to achieve this functionality, the idea is to load all target modules and they will be registered with registerTarget decorator, so it is less error prone than previous implementation.

testimage and testexport bbclasses were modified to support this. testexport required more work to find the targets and copy them into the exporting directory. The executor also needed to remove current choices (simpleremote, qemu) because now we can have custom targets.

Signed-off-by Mariano Lopez <[email protected]>

---
 meta/classes/testexport.bbclass   |  19 ++++++-
 meta/classes/testimage.bbclass    |   4 +-
 meta/lib/oeqa/core/target/base.py |  26 +++++++++
 meta/lib/oeqa/runtime/context.py  | 109 ++++++++------------------------------
 4 files changed, 68 insertions(+), 90 deletions(-)

meta/classes/testexport.bbclass +17 -2

index 56edda9..25d66ca 100644
@@ -58,8 +58,8 @@ def testexport_main(d):
     logger = logging.getLogger("BitBake")
 
     target = OERuntimeTestContextExecutor.getTarget(
-        d.getVar("TEST_TARGET"), None, d.getVar("TEST_TARGET_IP"),
-        d.getVar("TEST_SERVER_IP"))
+        d.getVar("TEST_TARGET"), d.getVar('BBLAYERS').split(), None,
+        d.getVar("TEST_TARGET_IP"), d.getVar("TEST_SERVER_IP"))
 
     host_dumper = OERuntimeTestContextExecutor.getHostDumper(
         d.getVar("testimage_dump_host"), d.getVar("TESTIMAGE_DUMP_DIR"))

@@ -108,6 +108,21 @@ def copy_needed_files(d, tc):
         else:
             shutil.copy2(src, dst)
 
+    # Get all target files.
+    t_path = os.path.join('lib', 'oeqa', 'core', 'target')
+    targets_to_copy = [os.path.join(root, filename)
+                       for path in d.getVar('BBLAYERS').split()
+                       for root, _, files in os.walk(os.path.join(path, t_path))
+                       for filename in files
+                       if filename.endswith('.py') ]
+
+    # Copy all targets.
+    export_target_path = os.path.join(export_path, t_path)
+    oe.path.remove(export_target_path)
+    bb.utils.mkdirhier(export_target_path)
+    for f in targets_to_copy:
+        shutil.copy2(f, export_target_path)
+
     # Remove cases and just copy the ones specified
     cases_path = os.path.join(export_path, 'lib', 'oeqa', 'runtime', 'cases')
     oe.path.remove(cases_path)

meta/classes/testimage.bbclass +2 -2

index bfe5714..3199ac4 100644
@@ -242,8 +242,8 @@ def testimage_main(d):
 
     # the robot dance
     target = OERuntimeTestContextExecutor.getTarget(
-        d.getVar("TEST_TARGET"), None, d.getVar("TEST_TARGET_IP"),
-        d.getVar("TEST_SERVER_IP"), **target_kwargs)
+        d.getVar("TEST_TARGET"), d.getVar('BBLAYERS').split(), None,
+        d.getVar("TEST_TARGET_IP"), d.getVar("TEST_SERVER_IP"), **target_kwargs)
 
     # test context
     tc = OERuntimeTestContext(td, logger, target, host_dumper,

meta/lib/oeqa/core/target/base.py +26 -0

index c886491..426a577 100644
@@ -54,3 +54,29 @@ class OETarget(object):
     @abstractmethod
     def copyDirTo(self, localSrc, remoteDst):
         pass
+
+def discover_targets(layer_paths):
+    """
+    Imports modules found in 'lib/oeqa/core/target'.
+
+    This is used to register targets using registerTarget decorator.
+    """
+
+    target_path = 'lib/oeqa/core/target'
+    paths = [os.path.join(p, target_path) for p in layer_paths]
+    for path in paths:
+        files_python = [os.path.join(root, filename)
+                        for root, _, filenames in os.walk(path)
+                        for filename in filenames
+                        if filename.endswith('.py')]
+        for f in files_python:
+            if '__init__.py' in f:
+                continue
+            abs_path = os.path.abspath(f)
+            for sys_path in sys.path:
+                if sys_path in abs_path:
+                    rel_path = os.path.relpath(abs_path, sys_path)
+                    break
+
+            name = rel_path.replace('.py', '').replace(os.path.sep, '.')
+            importlib.import_module(name)

meta/lib/oeqa/runtime/context.py +23 ~~-86~~

139 lines not on this page · raw message

Signature · 1 line

2.10.2

Richard Purdie

2017-03-27 13:38 UTC #4 lore raw

In reply to [email protected]

Quoted 8 lines · On Fri, 2017-03-24 at 15:06 -0700, [email protected] wrote:

From: Mariano Lopez <[email protected]>
This series allows to import custom targets to be used with runtime testing.
Now is possible to have a target in <layer>/lib/oeqa/core/target and testimage and testexport can use such target using TEST_TARGET variable.

Quoted 5 lines

To register a custom target you just need use decorate a target class with
registerTarget, and set "targetName" attribute to the name that will be used
by TEST_TARGET variable.

Unfortunately something about this series break oe-selftest:

https://autobuilder.yocto.io/builders/nightly-oe-selftest/builds/229/st
eps/Running%20oe-selftest/logs/stdio

It doesn't do it in isolation, that test run by itself is fine. When run as part of oe-selftest it breaks though. I did bisect it down to this series and things have all built fine after I dropped these 3 patches.

So we'll have to figure out what is happening here...

Cheers,

Richard

Mariano Lopez

author

2017-03-27 15:20 UTC #5 lore raw

Quoted 19 lines · Richard Purdie

Quoted 8 lines · On Fri, 2017-03-24 at 15:06 -0700, [email protected] wrote:
From: Mariano Lopez <[email protected]>
This series allows to import custom targets to be used with runtime testing.
Now is possible to have a target in <layer>/lib/oeqa/core/target and testimage and testexport can use such target using TEST_TARGET variable.
Quoted 5 lines
To register a custom target you just need use decorate a target class with
registerTarget, and set "targetName" attribute to the name that will be used
by TEST_TARGET variable.
Unfortunately something about this series break oe-selftest:
https://autobuilder.yocto.io/builders/nightly-oe-selftest/builds/229/st
eps/Running%20oe-selftest/logs/stdio

Quoted 6 lines

It doesn't do it in isolation, that test run by itself is fine. When run as part of oe-selftest it breaks though. I did bisect it down to this series and things have all built fine after I dropped these 3 patches.
So we'll have to figure out what is happening here...

I'm very sorry that you have to do a bisect to find this out :(

I did run selftest in my workstation to avoid this thing but it seems it wasn't enough.

The problem is when importing the modules with
oe-test (outside bitbake). I did check it with python 3.4 but I didn't test it with 3.5 so, I'll start with that.

Regards,
Mariano

Quoted 4 lines

Cheers,
Richard