Enable DAFT support into the AB

PATCH In review Aníbal Limón 2 participants 7 messages Jul 18, 2017 – Jul 19, 2017

lore

Share & tools

lore

Aníbal Limón

author

2017-07-18 16:14 UTC #0 lore raw

v3 cover Enable DAFT support into the AB

This v3 address the following issues,

* Uses BuildImages property instead of uppercase to avoid confusion with BB variables.
* Refactor code to get DUT configuration in lib/daft.py.
* Remove the sudo usage when executing daft flash.

Also the changes can be reviewed at,

http://git.yoctoproject.org/cgit/cgit.cgi/yocto-autobuilder/log/?h=contrib/alimon/daft

Aníbal Limón (5):
buildsteps/BuildImages: Setproperty BuildImages buildsteps/RunSanityTests: Add support for simpleremote target autobuilder/buildsteps: Add DaftGetDevices step autobuilder/buildsteps: Add DAFTFlash buildstep buildset-config.yocto-qa: Add intel-corei7-64-daft buildset

 buildset-config.yocto-qa/intel-corei7-64-daft.conf |  59 +++++++++++
 config/autobuilder.conf.example                    |   4 +
 .../autobuilder/buildsteps/BuildImages.py          |   3 +
 .../autobuilder/buildsteps/DaftFlash.py            |  39 +++++++
 .../autobuilder/buildsteps/DaftGetDevices.py       |  44 ++++++++
 .../autobuilder/buildsteps/RunSanityTests.py       |   8 ++
 lib/python2.7/site-packages/autobuilder/config.py  |   2 +
 .../site-packages/autobuilder/lib/daft.py          | 118 +++++++++++++++++++++
 8 files changed, 277 insertions(+)
 create mode 100644 buildset-config.yocto-qa/intel-corei7-64-daft.conf
 create mode 100644 lib/python2.7/site-packages/autobuilder/buildsteps/DaftFlash.py
 create mode 100644 lib/python2.7/site-packages/autobuilder/buildsteps/DaftGetDevices.py
 create mode 100644 lib/python2.7/site-packages/autobuilder/lib/daft.py

Signature · 1 line

2.11.0

Aníbal Limón

author

2017-07-18 16:14 UTC #1 lore raw

PATCH v3 1/5 buildsteps/BuildImages: Setproperty BuildImages

In certain situations is need to know what images was built for example the DAFT executes flashing over certain image.

[YOCTO #10604]

Signed-off-by Aníbal Limón <[email protected]>

---
 lib/python2.7/site-packages/autobuilder/buildsteps/BuildImages.py | 3 +++
 1 file changed, 3 insertions(+)

lib/python2.7/site-packages/autobuilder/buildsteps/BuildImages.py +3 -0

index f276863f8ff..8f3a5c04f73 100644
@@ -172,6 +172,9 @@ class BuildImages(BitbakeShellCommand):
                     self.description = ["Building " + str(self.images)]
         if self.forcebash:
             self.command = ['bash', '-c', self.command]
+        
+        self.setProperty("BuildImages", self.images)
+
         ShellCommand.start(self)
 
     def describe(self, done=False):

Signature · 1 line

2.11.0

Aníbal Limón

author

2017-07-18 16:14 UTC #2 lore raw

PATCH v3 2/5 buildsteps/RunSanityTests: Add support for simpleremote target

In reply to Aníbal Limón

The testimage class supports a simpleremote target specifying the server and target ip to run tests over SSH.

The main usage of this will in conjunction with the DAFT to make tests on real HW.

[YOCTO #10604]

Signed-off-by Aníbal Limón <[email protected]>

---
 .../site-packages/autobuilder/buildsteps/RunSanityTests.py        | 8 ++++++++
 1 file changed, 8 insertions(+)

lib/python2.7/site-packages/autobuilder/buildsteps/RunSanityTests.py +8 -0

index 48cd029fb71..c52672e9cc4 100644
@@ -64,6 +64,14 @@ class RunSanityTests(BitbakeShellCommand):
         elif layerversion > 1:
             self.command = self.command + "echo 'INHERIT += \"testimage\"' >> ./conf/auto.conf;"
             self.command = self.command + "echo 'TEST_QEMUBOOT_TIMEOUT = \"1500\"' >> ./conf/auto.conf;"
+
+            target_ip = self.getProperty('target_ip')
+            server_ip = self.getProperty('server_ip')
+            if target_ip and server_ip:
+                self.command = self.command +  "echo 'TEST_TARGET=\"simpleremote\"' >> ./conf/auto.conf;"
+                self.command = self.command +  "echo 'TEST_TARGET_IP=\"%s\"' >> ./conf/auto.conf;" % target_ip
+                self.command = self.command +  "echo 'TEST_SERVER_IP=\"%s\"' >> ./conf/auto.conf;" % server_ip
+
             if self.suites:
                 if layerversion > 3:
                     self.command = self.command + "echo 'TEST_SUITES = \"" + self.suites + "\"'" + " >> ./conf/auto.conf;"

Signature · 1 line

2.11.0

Aníbal Limón

author

2017-07-18 16:14 UTC #3 lore raw

PATCH v3 3/5 autobuilder/buildsteps: Add DaftGetDevices step

In reply to Aníbal Limón

This buildstep will copy the devices configuration from the worker and read it, in order to get daft ip an ssh port, to use later in RunSanityTests

[YOCTO #10604]

Signed-off-by Aníbal Limón <[email protected]>

Signed-off-by Edwin Plauchu <[email protected]>

---
 config/autobuilder.conf.example                    |   3 +
 .../autobuilder/buildsteps/DaftGetDevices.py       |  44 ++++++++
 lib/python2.7/site-packages/autobuilder/config.py  |   1 +
 .../site-packages/autobuilder/lib/daft.py          | 118 +++++++++++++++++++++
 4 files changed, 166 insertions(+)
 create mode 100644 lib/python2.7/site-packages/autobuilder/buildsteps/DaftGetDevices.py
 create mode 100644 lib/python2.7/site-packages/autobuilder/lib/daft.py

config/autobuilder.conf.example +3 -0

index 9e2fa8934e6..565a1a31b30 100644
@@ -96,3 +96,6 @@ PERFORMANCE_MAIL_TO = "root@localhost otherperson@localhost"
 PERFORMANCE_MAIL_CC = "buildcc@localhost"
 PERFORMANCE_MAIL_BCC = "buildbcc@localhost"
 PERFORMANCE_MAIL_SIG = "Multiline\nSig\nLine"
+
+[Daft]
+DAFT_WORKER_DEVICES_CFG = "/etc/daft/devices.cfg"

lib/python2.7/site-packages/autobuilder/buildsteps/DaftGetDevices.py new file +44 -0

44 lines not on this page · raw message

lib/python2.7/site-packages/autobuilder/config.py +1 -0

index 9d945b17525..5bcf6c6ff9d 100644
@@ -23,3 +23,4 @@ RPM_PUBLISH_DIR = os.environ.get("RPM_PUBLISH_DIR")
 IPK_PUBLISH_DIR = os.environ.get("IPK_PUBLISH_DIR")
 DEB_PUBLISH_DIR = os.environ.get("DEB_PUBLISH_DIR")
 PERFORMANCE_PUBLISH_DIR = os.environ.get("PERFORMANCE_PUBLISH_DIR")
+DAFT_WORKER_DEVICES_CFG = os.environ.get("DAFT_WORKER_DEVICES_CFG")

lib/python2.7/site-packages/autobuilder/lib/daft.py new file +118 -0

118 lines not on this page · raw message

Signature · 1 line

2.11.0

Aníbal Limón

author

2017-07-18 16:14 UTC #4 lore raw

PATCH v3 4/5 autobuilder/buildsteps: Add DAFTFlash buildstep

In reply to Aníbal Limón

This will execute daft flash cycle without reboot and prepare the dut for sanity tests.

[YOCTO #10604]

Signed-off-by Aníbal Limón <[email protected]>

Signed-off-by Edwin Plauchu <[email protected]>

---
 config/autobuilder.conf.example                    |  1 +
 .../autobuilder/buildsteps/DaftFlash.py            | 39 ++++++++++++++++++++++
 lib/python2.7/site-packages/autobuilder/config.py  |  1 +
 3 files changed, 41 insertions(+)
 create mode 100644 lib/python2.7/site-packages/autobuilder/buildsteps/DaftFlash.py

config/autobuilder.conf.example +1 -0

index 565a1a31b30..2bda57add83 100644
@@ -99,3 +99,4 @@ PERFORMANCE_MAIL_SIG = "Multiline\nSig\nLine"
 
 [Daft]
 DAFT_WORKER_DEVICES_CFG = "/etc/daft/devices.cfg"
+DAFT_WORKER_WORKSPACE = "/home/ab/workspace"

lib/python2.7/site-packages/autobuilder/buildsteps/DaftFlash.py new file +39 -0

new file mode 100644
index 00000000000..daf9c374398
@@ -0,0 +1,39 @@
+import os
+
+from autobuilder.config import DAFT_WORKER_WORKSPACE
+from buildbot.steps.shell import ShellCommand
+
+class DaftFlash(ShellCommand):
+    haltOnFailure = True
+
+    name = "DaftFlash"
+
+    def __init__(self, factory, argdict=None, **kwargs):
+        self.tests = None
+        self.factory = factory
+        for k, v in argdict.iteritems():
+            setattr(self, k, v)
+        self.description = "DUT Flashing"
+        self.timeout = 100000
+        kwargs['timeout']=self.timeout
+        ShellCommand.__init__(self, **kwargs)
+
+    def start(self):
+        workspace_dir = DAFT_WORKER_WORKSPACE
+
+        dut_name = self.getProperty('dut_name')
+        workdir = self.getProperty('workdir')
+        image = self.getProperty('BuildImages')
+        machine = self.getProperty('MACHINE')
+
+        image_path = os.path.join(workdir, 'build', 'build', 'tmp', 'deploy',
+                'images', machine, '%s-%s.hddimg' % (image, machine))
+        image_name = os.path.basename(image_path)
+
+        # XXX: DAFT needs to have the image in a shared workspace 
+        self.command = "cp %s %s; " % (image_path, workspace_dir)
+        self.command += "cd %s; " % (workspace_dir)
+        daft_cmd = "daft --setout %s %s" % (dut_name, image_name)
+        self.command += "cd {} && {} ".format(workspace_dir, daft_cmd)
+
+        ShellCommand.start(self)

lib/python2.7/site-packages/autobuilder/config.py +1 -0

index 5bcf6c6ff9d..fcc08f0a591 100644
@@ -24,3 +24,4 @@ IPK_PUBLISH_DIR = os.environ.get("IPK_PUBLISH_DIR")
 DEB_PUBLISH_DIR = os.environ.get("DEB_PUBLISH_DIR")
 PERFORMANCE_PUBLISH_DIR = os.environ.get("PERFORMANCE_PUBLISH_DIR")
 DAFT_WORKER_DEVICES_CFG = os.environ.get("DAFT_WORKER_DEVICES_CFG")
+DAFT_WORKER_WORKSPACE = os.environ.get("DAFT_WORKER_WORKSPACE")

Signature · 1 line

2.11.0

Aníbal Limón

author

2017-07-18 16:14 UTC #5 lore raw

PATCH v3 5/5 buildset-config.yocto-qa: Add intel-corei7-64-daft buildset

In reply to Aníbal Limón

This buildset is for support running sanity tests over a DUT like Minnowboard for the moment only builds core-image-sato, there is a need to make it configurable.

Also the listo of the available devices are fixed into the UI as parameters.

[YOCTO #10604]

Signed-off-by Aníbal Limón <[email protected]>

---
 buildset-config.yocto-qa/intel-corei7-64-daft.conf | 59 ++++++++++++++++++++++
 1 file changed, 59 insertions(+)
 create mode 100644 buildset-config.yocto-qa/intel-corei7-64-daft.conf

buildset-config.yocto-qa/intel-corei7-64-daft.conf new file +59 -0

new file mode 100644
index 00000000000..cb3bcdade75
@@ -0,0 +1,59 @@
+[intel-corei7-64-daft]
+builders: 'example-worker'
+repos: [{'poky':
+            {'repourl':'git://git.yoctoproject.org/poky',
+             'layerversion':{'core':'meta', 'yoctobsp':'meta-yocto-bsp', 'yocto':'meta-yocto', 'poky':'meta-poky'},
+             'branch':'master'}},
+        {'meta-intel':
+            {'repourl':'git://git.yoctoproject.org/meta-intel',
+             'layerversion':{'intel':'meta-intel'},
+             'branch':'master'}}]
+props: [{'kmeta':{'prop_type':'StringParameter',
+                       'size': 15,
+                       'name': 'kmeta',
+                       'default': '',
+                       'label':'<hr><h3> branch for KMETA_${MACHINE} (default is blank):</h3>'}},
+        {'srcrev_meta':{'prop_type':'StringParameter',
+                       'size': 15,
+                       'name': 'srcrev_meta',
+                       'default': '${AUTOREV}',
+                       'label':'<hr><h3> SRCREV for KMETA_${MACHINE} (default is ${AUTOREV}):<br></h3>'}},
+        {'srcuri_meta':{'prop_type':'StringParameter',
+                       'size': 50,
+                       'name': 'srcuri_meta',
+                       'default': '',
+                       'label':'<hr><h3> SRC_URI_pn-linux-yocto KMETA append. (default is blank).<br> This should be constructed as *just* the git uri portion of the string. <br>(git://git.yoctoproject.org/linux-yocto-3.19.git</h3>'}},
+        {'kbranch':{'prop_type':'StringParameter',
+                       'size': 15,
+                       'name': 'kbranch',
+                       'default': '',
+                       'label':'<hr><h3> branch for KBRANCH_${MACHINE} (default is blank):</h3>'}},
+        {'srcrev_machine':{'prop_type':'StringParameter',
+                       'size': 15,
+                       'name': 'srcrev_machine',
+                       'default': '${AUTOREV}',
+                       'label':'<hr><h3> SRCREV for KBRANCH. (default is ${AUTOREV}):<br></h3>'}},
+        {'srcuri_machine':{'prop_type':'StringParameter',
+                       'size': 50,
+                       'name': 'srcuri_machine',
+                       'default': '',
+                       'label':'<hr><h3> SRC_URI_pn-linux-yocto KBRANCH append. (default is blank).<br> This should be constructed as *just* the git uri portion of the string. <br>(git://git.yoctoproject.org/linux-yocto-3.19.git</h3>'}},
+	{'dut':{'prop_type':'ChoiceStringParameter',
+		   'choices': ['Minnowboard1'],
+		   'name': 'dut',
+		   'label':'<hr><h3>Selects DUT for flashing.<br></h3>'}}]
+steps: [{'SetDest':{}},
+        {'CheckOutLayers': {}},
+        {'RunPreamble': {}},
+        {'CheckBSPExists': {}},
+        {'GetDistroVersion' : {'distro': 'poky'}},
+        {'CreateAutoConf': {'machine': 'intel-corei7-64', 'SDKMACHINE' : 'x86_64', 
+                            'distro': 'poky'}},
+        {'CreateBBLayersConf': {'buildprovider' : 'yocto',
+                                'bsplayer': True, 'bspprovider': 'intel',
+                                'layerdirs': ['meta-intel', 'meta-intel/meta-tlk']}},
+        {'AddKernelProps': {}},
+        {'DaftGetDevices': {}},
+        {'BuildImages': {'images': 'core-image-sato'}},
+        {'DaftFlash':{}},
+        {'RunSanityTests': {'images': 'core-image-sato'}},]

Signature · 1 line

2.11.0

Victor Rodriguez

2017-07-19 11:07 UTC #6 lore raw

Re: [yocto-autobuilder][PATCHv3 5/5] buildset-config.yocto-qa: Add intel-corei7-64-daft buildset

On Tue, Jul 18, 2017 at 11:14 AM, Aníbal Limón <[email protected]

Quoted 100 lines

wrote:
This buildset is for support running sanity tests over a DUT like Minnowboard for the moment only builds core-image-sato, there is a need to make it configurable.
Also the listo of the available devices are fixed into the UI as parameters.
[YOCTO #10604]
Signed-off-by Aníbal Limón <[email protected]>
---
 buildset-config.yocto-qa/intel-corei7-64-daft.conf | 59
++++++++++++++++++++++
 1 file changed, 59 insertions(+)
 create mode 100644 buildset-config.yocto-qa/intel-corei7-64-daft.conf
buildset-config.yocto-qa/intel-corei7-64-daft.conf new file +5 -0
new file mode 100644
index 00000000000..cb3bcdade75
@@ -0,0 +1,59 @@
+[intel-corei7-64-daft]
+builders: 'example-worker'
+repos: [{'poky':
+            {'repourl':'git://git.yoctoproject.org/poky',
+             'layerversion':{'core':'meta', 'yoctobsp':'meta-yocto-bsp',
'yocto':'meta-yocto', 'poky':'meta-poky'},
buildset-config.yocto-qa/intel-corei7-64-daft.conf new file +10 -0
+             'branch':'master'}},
+        {'meta-intel':
+            {'repourl':'git://git.yoctoproject.org/meta-intel',
+             'layerversion':{'intel':'meta-intel'},
+             'branch':'master'}}]
+props: [{'kmeta':{'prop_type':'StringParameter',
+                       'size': 15,
+                       'name': 'kmeta',
+                       'default': '',
+                       'label':'<hr><h3> branch for KMETA_${MACHINE}
(default is blank):</h3>'}},
buildset-config.yocto-qa/intel-corei7-64-daft.conf new file +5 -0
+        {'srcrev_meta':{'prop_type':'StringParameter',
+                       'size': 15,
+                       'name': 'srcrev_meta',
+                       'default': '${AUTOREV}',
+                       'label':'<hr><h3> SRCREV for KMETA_${MACHINE}
(default is ${AUTOREV}):<br></h3>'}},
buildset-config.yocto-qa/intel-corei7-64-daft.conf new file +5 -0
+        {'srcuri_meta':{'prop_type':'StringParameter',
+                       'size': 50,
+                       'name': 'srcuri_meta',
+                       'default': '',
+                       'label':'<hr><h3> SRC_URI_pn-linux-yocto KMETA
append. (default is blank).<br> This should be constructed as *just* the git uri portion of the string. <br>(git://git.yoctoproject. org/linux-yocto-3.19.git</h3>'}},
buildset-config.yocto-qa/intel-corei7-64-daft.conf new file +5 -0
+        {'kbranch':{'prop_type':'StringParameter',
+                       'size': 15,
+                       'name': 'kbranch',
+                       'default': '',
+                       'label':'<hr><h3> branch for KBRANCH_${MACHINE}
(default is blank):</h3>'}},
buildset-config.yocto-qa/intel-corei7-64-daft.conf new file +5 -0
+        {'srcrev_machine':{'prop_type':'StringParameter',
+                       'size': 15,
+                       'name': 'srcrev_machine',
+                       'default': '${AUTOREV}',
+                       'label':'<hr><h3> SRCREV for KBRANCH. (default is
${AUTOREV}):<br></h3>'}},
buildset-config.yocto-qa/intel-corei7-64-daft.conf new file +5 -0
+        {'srcuri_machine':{'prop_type':'StringParameter',
+                       'size': 50,
+                       'name': 'srcuri_machine',
+                       'default': '',
+                       'label':'<hr><h3> SRC_URI_pn-linux-yocto KBRANCH
append. (default is blank).<br> This should be constructed as *just* the git uri portion of the string. <br>(git://git.yoctoproject. org/linux-yocto-3.19.git</h3>'}},
buildset-config.yocto-qa/intel-corei7-64-daft.conf new file +10 -0
+       {'dut':{'prop_type':'ChoiceStringParameter',
+                  'choices': ['Minnowboard1'],
+                  'name': 'dut',
+                  'label':'<hr><h3>Selects DUT for flashing.<br></h3>'}}]
+steps: [{'SetDest':{}},
+        {'CheckOutLayers': {}},
+        {'RunPreamble': {}},
+        {'CheckBSPExists': {}},
+        {'GetDistroVersion' : {'distro': 'poky'}},
+        {'CreateAutoConf': {'machine': 'intel-corei7-64', 'SDKMACHINE' :
'x86_64',
buildset-config.yocto-qa/intel-corei7-64-daft.conf new file +4 -0
+                            'distro': 'poky'}},
+        {'CreateBBLayersConf': {'buildprovider' : 'yocto',
+                                'bsplayer': True, 'bspprovider': 'intel',
+                                'layerdirs': ['meta-intel',
'meta-intel/meta-tlk']}},
buildset-config.yocto-qa/intel-corei7-64-daft.conf new file +5 -1
+        {'AddKernelProps': {}},
+        {'DaftGetDevices': {}},
+        {'BuildImages': {'images': 'core-image-sato'}},
+        {'DaftFlash':{}},
+        {'RunSanityTests': {'images': 'core-image-sato'}},]
--
2.11.0

Quoted 1 line

Is there any documentation for this , i like a lot the idea and i would

like to know how to test this

Quoted 6 lines

--
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto