Skip to contentSkip to conversation

New buildsets and options for 1.5

PATCH In review Stefan Stanacar 2 participants 8 messages Apr 24, 2013 – Apr 27, 2013

lore
Share & tools
lore
#0 lore raw

cover New buildsets and options for 1.5

Hello Beth,

Here are the new changes, I've fixed that small issue, hope it looks good now. After some more tests, I'll follow up with some builds for ipk only images :), but I'm getting ahead of myelf now.

Regards,
Stefan

The following changes since commit 0178163efc6670c9acb982dd579671396cb24290:

PublishLayerTarballs.py: bad redirect. (2013-04-22 22:56:58 -0700)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib stefans/yab0
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=stefans/yab0

Stefan Stanacar (6):
nightly-qa-extras.conf: add new buildset config CreateAutoConf.py: add initmgr option buildset-config/nightly-qa-systemd.conf: add a new target for systemd
images
nightly.conf, yoctoAB.conf: enable the new buildsets buildsteps/RunSanityTests.py: add suport for custom steps CreateAutoConf.py: add option for package types

 buildset-config/nightly-qa-extras.conf             | 16 ++++++++++++++++
 buildset-config/nightly-qa-systemd.conf            | 20 ++++++++++++++++++++
 buildset-config/nightly.conf                       |  3 ++-
 buildset-config/yoctoAB.conf                       |  3 ++-
 .../autobuilder/buildsteps/CreateAutoConf.py       | 22 ++++++++++++++++++++--
 .../autobuilder/buildsteps/RunSanityTests.py       |  3 +++
 6 files changed, 63 insertions(+), 4 deletions(-)
 create mode 100644 buildset-config/nightly-qa-extras.conf
 create mode 100644 buildset-config/nightly-qa-systemd.conf
Signature · 1 line
1.8.1.4
#1 lore raw

PATCH 1/6 nightly-qa-extras.conf: add new buildset config

This buildset is for extra images which don't really fit in other buildsets. For now it includes builds for two images: one with btrfs and diffrent root home dir and a sato image with read-only rootfs.

---
 buildset-config/nightly-qa-extras.conf | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
 create mode 100644 buildset-config/nightly-qa-extras.conf
buildset-config/nightly-qa-extras.conf new file +16 -0
new file mode 100644
index 0000000..4098f53
@@ -0,0 +1,16 @@
+[nightly-qa-extras]
+builders: 'builder1'
+repos: [{'poky':
+            {'repourl':'git://git.yoctoproject.org/poky',
+             'bbpriority':'1',
+             'branch':'master'}}]
+steps: [{'SetDest':{}},
+        {'CheckOutLayers': {}},
+        {'RunPreamble': {}},
+        {'GetDistroVersion' : {'distro': 'poky'}},
+        {'CreateAutoConf': {'machine': 'qemux86-64', 'SDKMACHINE' : 'i686', 'distro': 'poky', 'atextappend' : '\nROOT_HOME = "/root"\nIMAGE_FSTYPES = "btrfs"\nKERNEL_FEATURES_append = " cfg/fs/btrfs"\n'}},
+        {'CreateBBLayersConf': {'buildprovider' : 'yocto'}},
+        {'BuildImages': {'images': 'core-image-sato'}},
+        {'RunSanityTests': {'images': 'core-image-sato'}},
+        {'CreateAutoConf': {'machine': 'qemux86-64', 'SDKMACHINE' : 'i686', 'distro': 'poky', 'atextappend' : '\nEXTRA_IMAGE_FEATURES = "debug-tweaks read-only-rootfs"\n'}},
+        {'BuildImages': {'images': 'core-image-sato'}}]
Signature · 1 line
1.8.1.4
#4 lore raw

PATCH 4/6 nightly.conf, yoctoAB.conf: enable the new buildsets

In reply to Stefan Stanacar

Add the new nightly-qa-extras and nightly-qa-systemd buildsets to the nightly target.

---
 buildset-config/nightly.conf | 3 ++-
 buildset-config/yoctoAB.conf | 3 ++-
 2 files changed, 4 insertions(+), 2 deletions(-)
buildset-config/nightly.conf +2 -1
index 58a8531..ffbc271 100644
@@ -67,7 +67,8 @@ steps: [{'SetDest':{}},
                               'nightly-x32': {},
                               'nightly-x86-64': {}, 'nightly-x86-64-lsb': {},
                               'nightly-x86': {}, 'nightly-x86-lsb': {},
-                              'poky-tiny': {}}}},
+                              'poky-tiny': {},
+                              'nightly-qa-extras': {}, 'nightly-qa-systemd': {}}}},
         {'PrepPkgIndex' : {}},
         {'BuildImages': {'images': 'package-index'}},
         {'CreateAutoConf': {'machine': 'atom-pc', 'SDKMACHINE' : 'x86_64', 
buildset-config/yoctoAB.conf +2 -1
index 5f29400..2f22767 100644
@@ -4,4 +4,5 @@ order: ['nightly', 'eclipse-plugin', 'nightly-arm', 'nightly-arm-lsb', 'nightly-
         'nightly-x86-64', 'nightly-x86-64-lsb', 'nightly-x86',
         'nightly-x86-lsb', 'nightly-x32', 'nightly-multilib',
         'nightly-world', 'nightly-non-gpl3', 'nightly-oecore',
-        'nightly-intel-gpl', 'poky-tiny', 'build-appliance']
+        'nightly-intel-gpl', 'poky-tiny', 'build-appliance',
+        'nightly-qa-extras', 'nightly-qa-systemd']
Signature · 1 line
1.8.1.4
#5 lore raw

PATCH 5/6 buildsteps/RunSanityTests.py: add suport for custom steps

In reply to Stefan Stanacar

Add an option so we can pass custom qemuimage tests, by appending TEST_SCEN to local.conf.

This isn't used by any buildsets right now.

It could be used when we want to run only certain sanity tests (i.e don't run all the tests in the image scenario but run only sanity:boot and sanity:dmesg by appending TEST_SCENE = "sanity:boot sanity:dmesg")

---
 lib/python2.7/site-packages/autobuilder/buildsteps/RunSanityTests.py | 3 +++
 1 file changed, 3 insertions(+)
lib/python2.7/site-packages/autobuilder/buildsteps/RunSanityTests.py +3 -0
index a352de5..1a428be 100644
@@ -23,6 +23,7 @@ class RunSanityTests(ShellCommand):
     def __init__(self, factory, argdict=None, **kwargs):
         self.factory = factory
         self.images=""
+        self.scene=None
         for k, v in argdict.iteritems():
             setattr(self, k, v)
         self.description = "Running Sanity Tests"
@@ -31,6 +32,8 @@ class RunSanityTests(ShellCommand):
         command = command + "(echo > /dev/tcp/localhost/5901) 2> /dev/null;"
         command = command + 'if [ $? -ne 0 ]; then echo "Starting a VNC server on :1"; vncserver :1; else echo "Will use a VNC server already running on :1"; fi;'
         command = command + "echo 'IMAGETEST = \"qemu\"' >> ./conf/auto.conf;"
+        if self.scene:
+            command = command + "echo 'TEST_SCEN = \"" + self.scene + "\"'" + " >> ./conf/auto.conf;"
         command = command + "DISPLAY=localhost:1 bitbake " + self.images + " -c qemuimagetest_standalone"
         # Timeout needs to be passed to LoggingBuildStep as a kwarg
         self.timeout = 100000
Signature · 1 line
1.8.1.4
#6 lore raw

PATCH 6/6 CreateAutoConf.py: add option for package types

In reply to Stefan Stanacar

Using this option we'll be able to select only certain packages to be built and because the order is important we determine the image type too. E.g 'packages' : 'deb rpm' will build deb and rpm packages but the image will use deb as package manager.

---
 .../site-packages/autobuilder/buildsteps/CreateAutoConf.py         | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)
lib/python2.7/site-packages/autobuilder/buildsteps/CreateAutoConf.py +6 -1
index fd73a3d..8fbef3e 100644
@@ -22,6 +22,7 @@ class CreateAutoConf(ShellCommand):
         self.machine=""
         self.buildapp=""
         self.distro="poky"
+        self.packages=None
         self.buildhistory=False
         self.gplv3=True
         self.multilib=False
@@ -59,7 +60,11 @@ class CreateAutoConf(ShellCommand):
         else:
             if self.distro != "oecore":
                 fout = fout + 'DISTRO = "' + self.distro + '"\n'
-            fout = fout + 'PACKAGE_CLASSES = "package_rpm package_deb package_ipk"\n'
+            if self.packages:
+                self.classes = [ "package_%s" % p for p in self.packages.split() ]
+                fout = fout + 'PACKAGE_CLASSES = "' + " ".join(self.classes) + '"\n'
+            else:
+                fout = fout + 'PACKAGE_CLASSES = "package_rpm package_deb package_ipk"\n'
             fout = fout + 'BB_NUMBER_THREADS = "' + os.environ.get('BB_NUMBER_THREADS') + '"\n'
             fout = fout + 'PARALLEL_MAKE = "-j ' + os.environ.get('PARALLEL_MAKE') + '"\n'
             fout = fout + 'SDKMACHINE ?= "' + self.SDKMACHINE + '"\n'
Signature · 1 line
1.8.1.4
#2 lore raw

PATCH 2/6 CreateAutoConf.py: add initmgr option

In reply to Stefan Stanacar

Add an option to set the init manager used. This is used for building images with systemd. The order used is important as it decides the primary init system.

---
 .../autobuilder/buildsteps/CreateAutoConf.py              | 15 ++++++++++++++-
 1 file changed, 14 insertions(+), 1 deletion(-)
lib/python2.7/site-packages/autobuilder/buildsteps/CreateAutoConf.py +14 -1
index 3a4b9f9..fd73a3d 100644
@@ -36,6 +36,7 @@ class CreateAutoConf(ShellCommand):
         self.adtdev=False
         self.factory = factory
         self.buildappsrcrev = "${AUTOREV}"
+        self.initmgr=None
         self.kwargs = kwargs
         for k, v in argdict.iteritems():
             if type(v) is bool:
@@ -112,13 +113,25 @@ class CreateAutoConf(ShellCommand):
                 fout = fout + 'BUILDHISTORY_COMMIT = "1"\n'
                 fout = fout + 'BUILDHISTORY_DIR = "' + os.environ.get('BUILD_HISTORY_DIR') + '/' + self.getProperty("buildername") + '/poky-buildhistory"\n'
                 fout = fout + 'BUILDHISTORY_PUSH_REPO = "' + os.environ.get('BUILD_HISTORY_REPO') + ' ' + self.getProperty("buildername") + ':' + self.getProperty("buildername") + '"\n'
-            if self.getProperty("branch_poky") != "danny" and self.getProperty("distroversion"):
+            if "danny" not in self.getProperty("branch_poky") and self.getProperty("distroversion"):
                 if self.adtdev == "True":
                     adtrepo_url=os.environ.get("ADTREPO_DEV_URL")
                     fout=fout+'ADTREPO = "' + adtrepo_url + '/' + self.getProperty("distroversion") + '-' + self.getProperty("got_revision_poky") + '-' + self.getProperty("branch_poky") + '"\n'
                 else:
                     adtrepo_url=os.environ.get("ADTREPO_URL")
                     fout=fout+'ADTREPO = "' + adtrepo_url + '/' + self.getProperty("distroversion") + '"\n'
+            if self.initmgr and "danny" not in self.getProperty("branch_poky"):
+                # we don't neet to test sysvinit only, because that's the default
+                if self.initmgr == "systemd":
+                    fout = fout + 'DISTRO_FEATURES_append = " systemd"\n'
+                    fout = fout + 'VIRTUAL-RUNTIME_init_manager = "systemd"\n'
+                    fout = fout + 'DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"\n'
+                if self.initmgr == "systemd sysvinit":
+                    fout = fout + 'DISTRO_FEATURES_append = " systemd"\n'
+                    fout = fout + 'VIRTUAL-RUNTIME_init_manager = "systemd"\n'
+                if self.initmgr == "sysvinit systemd":
+                    fout = fout + 'DISTRO_FEATURES_append = " systemd"\n'
+                    fout = fout + 'VIRTUAL-RUNTIME_init_manager = "sysvinit"\n'
         if self.atextappend:
             fout = fout + self.atextappend
 
Signature · 1 line
1.8.1.4
#3 lore raw

PATCH 3/6 buildset-config/nightly-qa-systemd.conf: add a new target for systemd images

In reply to Stefan Stanacar

Build systemd images in different configurations: systemd only, systemd+sysvinit and sysvinit+systemd (the order means which is the primary init system).

---
 buildset-config/nightly-qa-systemd.conf | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
 create mode 100644 buildset-config/nightly-qa-systemd.conf
buildset-config/nightly-qa-systemd.conf new file +20 -0
new file mode 100644
index 0000000..e35e6b1
@@ -0,0 +1,20 @@
+[nightly-qa-systemd]
+builders: 'builder1'
+repos: [{'poky':
+            {'repourl':'git://git.yoctoproject.org/poky',
+             'bbpriority':'1',
+             'branch':'master'}}]
+steps: [{'SetDest':{}},
+        {'CheckOutLayers': {}},
+        {'RunPreamble': {}},
+        {'GetDistroVersion' : {'distro': 'poky'}},
+        {'CreateAutoConf': {'machine': 'qemux86-64', 'SDKMACHINE' : 'i686', 'distro': 'poky', 'initmgr' : 'systemd sysvinit'}},
+        {'CreateBBLayersConf': {'buildprovider' : 'yocto'}},
+        {'BuildImages': {'images': 'core-image-sato'}},
+        {'RunSanityTests': {'images': 'core-image-sato'}},
+        {'CreateAutoConf': {'machine': 'qemux86-64', 'SDKMACHINE' : 'i686', 'distro': 'poky', 'initmgr' : 'sysvinit systemd'}},
+        {'BuildImages': {'images': 'core-image-sato'}},
+        {'RunSanityTests': {'images': 'core-image-sato'}},
+        {'CreateAutoConf': {'machine': 'qemux86-64', 'SDKMACHINE' : 'i686', 'distro': 'poky', 'initmgr' : 'systemd'}},
+        {'BuildImages': {'images': 'core-image-sato'}},
+        {'RunSanityTests': {'images': 'core-image-sato'}}]
Signature · 1 line
1.8.1.4
#7 lore raw
In reply to Stefan Stanacar

Merged into eflanagan/yocto-autobuilder-refactor. Will deploy to production this week. Thanks!

-b

Quoted 41 lines · Stefan Stanacar

Hello Beth,

Here are the new changes, I've fixed that small issue, hope it looks good now. After some more tests, I'll follow up with some builds for ipk only images :), but I'm getting ahead of myelf now.

Regards,
Stefan

The following changes since commit 0178163efc6670c9acb982dd579671396cb24290:

PublishLayerTarballs.py: bad redirect. (2013-04-22 22:56:58 -0700)

are available in the git repository at:

git://git.yoctoproject.org/poky-contrib stefans/yab0
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=stefans/yab0

Stefan Stanacar (6):
nightly-qa-extras.conf: add new buildset config CreateAutoConf.py: add initmgr option buildset-config/nightly-qa-systemd.conf: add a new target for systemd
images
nightly.conf, yoctoAB.conf: enable the new buildsets buildsteps/RunSanityTests.py: add suport for custom steps CreateAutoConf.py: add option for package types

 buildset-config/nightly-qa-extras.conf             | 16 ++++++++++++++++
 buildset-config/nightly-qa-systemd.conf            | 20 ++++++++++++++++++++
 buildset-config/nightly.conf                       |  3 ++-
 buildset-config/yoctoAB.conf                       |  3 ++-
 .../autobuilder/buildsteps/CreateAutoConf.py       | 22 ++++++++++++++++++++--
 .../autobuilder/buildsteps/RunSanityTests.py       |  3 +++
 6 files changed, 63 insertions(+), 4 deletions(-)
 create mode 100644 buildset-config/nightly-qa-extras.conf
 create mode 100644 buildset-config/nightly-qa-systemd.conf

--
1.8.1.4

Signature · 3 lines
Elizabeth Flanagan
Yocto Project
Build and Release