New buildsets and options for 1.5
cover New buildsets and options for 1.5
Hello Beth,
Would you mind taking a look on these changes? Obviously they are meant post-dylan release, but I sent them earlier so you can have time to review and request changes.
It would be great to have them in 1.5 and I plan on adding even more builds (especially for multilib images and ipk images which proved tricky with postinstalls and systemd).
Thanks,
Stefan
The following changes since commit 0178163efc6670c9acb982dd579671396cb24290:
PublishLayerTarballs.py: bad redirect. (2013-04-22 22:56:58 -0700)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib stefans/yab0
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=stefans/yab0
Stefan Stanacar (6):
nightly-qa-extras.conf: add new buildset config CreateAutoConf.py: add initmgr option buildset-config/nightly-qa-systemd.conf: add a new target for systemd
images
nightly.conf, yoctoAB.conf: enable the new buildsets buildsteps/RunSanityTests.py: add suport for custom steps CreateAutoConf.py: add option for package types
buildset-config/nightly-qa-extras.conf | 16 ++++++++++++++++ buildset-config/nightly-qa-systemd.conf | 20 ++++++++++++++++++++ buildset-config/nightly.conf | 3 ++- buildset-config/yoctoAB.conf | 3 ++- .../autobuilder/buildsteps/CreateAutoConf.py | 20 +++++++++++++++++++- .../autobuilder/buildsteps/RunSanityTests.py | 3 +++ 6 files changed, 62 insertions(+), 3 deletions(-) create mode 100644 buildset-config/nightly-qa-extras.conf create mode 100644 buildset-config/nightly-qa-systemd.conf
Signature · 1 line
1.8.1.4
RFC 1/6 nightly-qa-extras.conf: add new buildset config
This buildset is for extra images which don't really fit in other buildsets. For now it includes builds for two images: one with btrfs and diffrent root home dir and a sato image with read-only rootfs.
--- buildset-config/nightly-qa-extras.conf | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 buildset-config/nightly-qa-extras.conf
buildset-config/nightly-qa-extras.conf new file +16 -0
@@ -0,0 +1,16 @@ +[nightly-qa-extras] +builders: 'builder1' +repos: [{'poky': + {'repourl':'git://git.yoctoproject.org/poky', + 'bbpriority':'1', + 'branch':'master'}}] +steps: [{'SetDest':{}}, + {'CheckOutLayers': {}}, + {'RunPreamble': {}}, + {'GetDistroVersion' : {'distro': 'poky'}}, + {'CreateAutoConf': {'machine': 'qemux86-64', 'SDKMACHINE' : 'i686', 'distro': 'poky', 'atextappend' : '\nROOT_HOME = "/root"\nIMAGE_FSTYPES = "btrfs"\nKERNEL_FEATURES_append = " cfg/fs/btrfs"\n'}}, + {'CreateBBLayersConf': {'buildprovider' : 'yocto'}}, + {'BuildImages': {'images': 'core-image-sato'}}, + {'RunSanityTests': {'images': 'core-image-sato'}}, + {'CreateAutoConf': {'machine': 'qemux86-64', 'SDKMACHINE' : 'i686', 'distro': 'poky', 'atextappend' : '\nEXTRA_IMAGE_FEATURES = "debug-tweaks read-only-rootfs"\n'}}, + {'BuildImages': {'images': 'core-image-sato'}}]
Signature · 1 line
1.8.1.4
RFC 2/6 CreateAutoConf.py: add initmgr option
In reply to Stefan StanacarAdd an option to set the init manager used. This is used for building images with systemd.
The order is important as it decides the primary one.
--- .../site-packages/autobuilder/buildsteps/CreateAutoConf.py | 13 +++++++++++++ 1 file changed, 13 insertions(+)
lib/python2.7/site-packages/autobuilder/buildsteps/CreateAutoConf.py +13 -0
@@ -36,6 +36,7 @@ class CreateAutoConf(ShellCommand): self.adtdev=False self.factory = factory self.buildappsrcrev = "${AUTOREV}" + self.initmgr=None self.kwargs = kwargs for k, v in argdict.iteritems(): if type(v) is bool:
@@ -119,6 +120,18 @@ class CreateAutoConf(ShellCommand): else: adtrepo_url=os.environ.get("ADTREPO_URL") fout=fout+'ADTREPO = "' + adtrepo_url + '/' + self.getProperty("distroversion") + '"\n' + if self.getProperty("branch_poky") != "danny" and self.initmgr: + # we don't neet to test sysvinit only, because that's the default + if self.initmgr == "systemd": + fout = fout + 'DISTRO_FEATURES_append = " systemd"\n' + fout = fout + 'VIRTUAL-RUNTIME_init_manager = "systemd"\n' + fout = fout + 'DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"\n' + if self.initmgr == "systemd sysvinit": + fout = fout + 'DISTRO_FEATURES_append = " systemd"\n' + fout = fout + 'VIRTUAL-RUNTIME_init_manager = "systemd"\n' + if self.initmgr == "sysvinit systemd": + fout = fout + 'DISTRO_FEATURES_append = " systemd"\n' + fout = fout + 'VIRTUAL-RUNTIME_init_manager = "sysvinit"\n' if self.atextappend: fout = fout + self.atextappend
Signature · 1 line
1.8.1.4
RFC 3/6 buildset-config/nightly-qa-systemd.conf: add a new target for systemd images
In reply to Stefan StanacarBuild systemd images in different configurations: systemd only, systemd+sysvinit and sysvinit+systemd (the order means which is the primary init system).
--- buildset-config/nightly-qa-systemd.conf | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 buildset-config/nightly-qa-systemd.conf
buildset-config/nightly-qa-systemd.conf new file +20 -0
@@ -0,0 +1,20 @@ +[nightly-qa-systemd] +builders: 'builder1' +repos: [{'poky': + {'repourl':'git://git.yoctoproject.org/poky', + 'bbpriority':'1', + 'branch':'master'}}] +steps: [{'SetDest':{}}, + {'CheckOutLayers': {}}, + {'RunPreamble': {}}, + {'GetDistroVersion' : {'distro': 'poky'}}, + {'CreateAutoConf': {'machine': 'qemux86-64', 'SDKMACHINE' : 'i686', 'distro': 'poky', 'initmgr' : 'systemd sysvinit'}}, + {'CreateBBLayersConf': {'buildprovider' : 'yocto'}}, + {'BuildImages': {'images': 'core-image-sato'}}, + {'RunSanityTests': {'images': 'core-image-sato'}}, + {'CreateAutoConf': {'machine': 'qemux86-64', 'SDKMACHINE' : 'i686', 'distro': 'poky', 'initmgr' : 'sysvinit systemd'}}, + {'BuildImages': {'images': 'core-image-sato'}}, + {'RunSanityTests': {'images': 'core-image-sato'}}, + {'CreateAutoConf': {'machine': 'qemux86-64', 'SDKMACHINE' : 'i686', 'distro': 'poky', 'initmgr' : 'systemd'}}, + {'BuildImages': {'images': 'core-image-sato'}}, + {'RunSanityTests': {'images': 'core-image-sato'}}]
Signature · 1 line
1.8.1.4
RFC 4/6 nightly.conf, yoctoAB.conf: enable the new buildsets
In reply to Stefan StanacarAdd the new nightly-qa-extras and nightly-qa-systemd buildsets to the nightly target.
--- buildset-config/nightly.conf | 3 ++- buildset-config/yoctoAB.conf | 3 ++- 2 files changed, 4 insertions(+), 2 deletions(-)
buildset-config/nightly.conf +2 -1
@@ -67,7 +67,8 @@ steps: [{'SetDest':{}}, 'nightly-x32': {}, 'nightly-x86-64': {}, 'nightly-x86-64-lsb': {}, 'nightly-x86': {}, 'nightly-x86-lsb': {}, - 'poky-tiny': {}}}}, + 'poky-tiny': {}, + 'nightly-qa-extras': {}, 'nightly-qa-systemd': {}}}}, {'PrepPkgIndex' : {}}, {'BuildImages': {'images': 'package-index'}}, {'CreateAutoConf': {'machine': 'atom-pc', 'SDKMACHINE' : 'x86_64',
buildset-config/yoctoAB.conf +2 -1
@@ -4,4 +4,5 @@ order: ['nightly', 'eclipse-plugin', 'nightly-arm', 'nightly-arm-lsb', 'nightly- 'nightly-x86-64', 'nightly-x86-64-lsb', 'nightly-x86', 'nightly-x86-lsb', 'nightly-x32', 'nightly-multilib', 'nightly-world', 'nightly-non-gpl3', 'nightly-oecore', - 'nightly-intel-gpl', 'poky-tiny', 'build-appliance'] + 'nightly-intel-gpl', 'poky-tiny', 'build-appliance', + 'nightly-qa-extras', 'nightly-qa-systemd']
Signature · 1 line
1.8.1.4
RFC 5/6 buildsteps/RunSanityTests.py: add suport for custom steps
In reply to Stefan StanacarAdd an option so we can pass custom qemuimage tests, by appending TEST_SCEN to local.conf.
This isn't used by any buildsets right now.
It could be used when we want to run only certain sanity tests (i.e don't run all the tests in the image scenario but run only sanity:boot and sanity:dmesg by appending TEST_SCENE = "sanity:boot sanity:dmesg")
--- lib/python2.7/site-packages/autobuilder/buildsteps/RunSanityTests.py | 3 +++ 1 file changed, 3 insertions(+)
lib/python2.7/site-packages/autobuilder/buildsteps/RunSanityTests.py +3 -0
@@ -23,6 +23,7 @@ class RunSanityTests(ShellCommand): def __init__(self, factory, argdict=None, **kwargs): self.factory = factory self.images="" + self.scene=None for k, v in argdict.iteritems(): setattr(self, k, v) self.description = "Running Sanity Tests"
@@ -31,6 +32,8 @@ class RunSanityTests(ShellCommand): command = command + "(echo > /dev/tcp/localhost/5901) 2> /dev/null;" command = command + 'if [ $? -ne 0 ]; then echo "Starting a VNC server on :1"; vncserver :1; else echo "Will use a VNC server already running on :1"; fi;' command = command + "echo 'IMAGETEST = \"qemu\"' >> ./conf/auto.conf;" + if self.scene: + command = command + "echo 'TEST_SCEN = \"" + self.scene + "\"'" + " >> ./conf/auto.conf;" command = command + "DISPLAY=localhost:1 bitbake " + self.images + " -c qemuimagetest_standalone" # Timeout needs to be passed to LoggingBuildStep as a kwarg self.timeout = 100000
Signature · 1 line
1.8.1.4
RFC 6/6 CreateAutoConf.py: add option for package types
In reply to Stefan StanacarUsing this option we'll be able to select only certain packages to be built and because the order is important we determine the image type too. E.g 'packages' : 'deb rpm' will build deb and rpm packages but the image will use deb as package manager.
--- .../site-packages/autobuilder/buildsteps/CreateAutoConf.py | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
lib/python2.7/site-packages/autobuilder/buildsteps/CreateAutoConf.py +6 -1
@@ -22,6 +22,7 @@ class CreateAutoConf(ShellCommand): self.machine="" self.buildapp="" self.distro="poky" + self.packages=None self.buildhistory=False self.gplv3=True self.multilib=False
@@ -59,7 +60,11 @@ class CreateAutoConf(ShellCommand): else: if self.distro != "oecore": fout = fout + 'DISTRO = "' + self.distro + '"\n' - fout = fout + 'PACKAGE_CLASSES = "package_rpm package_deb package_ipk"\n' + if self.packages: + self.classes = [ "package_%s" % p for p in self.packages.split() ] + fout = fout + 'PACKAGE_CLASSES = "' + " ".join(self.classes) + '"\n' + else: + fout = fout + 'PACKAGE_CLASSES = "package_rpm package_deb package_ipk"\n' fout = fout + 'BB_NUMBER_THREADS = "' + os.environ.get('BB_NUMBER_THREADS') + '"\n' fout = fout + 'PARALLEL_MAKE = "-j ' + os.environ.get('PARALLEL_MAKE') + '"\n' fout = fout + 'SDKMACHINE ?= "' + self.SDKMACHINE + '"\n'
Signature · 1 line
1.8.1.4
Re: [yocto-autobuilder][RFC 2/6] CreateAutoConf.py: add initmgr option
In reply to Stefan StanacarQuoted 22 lines · Stefan Stanacar
Add an option to set the init manager used. This is used for building images with systemd.
The order is important as it decides the primary one.Signed-off-by Stefan Stanacar <[email protected]>--- .../site-packages/autobuilder/buildsteps/CreateAutoConf.py | 13 +++++++++++++ 1 file changed, 13 insertions(+)lib/python2.7/site-packages/autobuilder/buildsteps/CreateAutoConf.py +1
-0@@ -36,6 +36,7 @@ class CreateAutoConf(ShellCommand): self.adtdev=False self.factory = factory self.buildappsrcrev = "${AUTOREV}" + self.initmgr=None self.kwargs = kwargs for k, v in argdict.iteritems(): if type(v) is bool:
lib/python2.7/site-packages/autobuilder/buildsteps/CreateAutoConf.py class CreateAutoConf(ShellCommand): · line 123
Quoted 4 lines
lib/python2.7/site-packages/autobuilder/buildsteps/CreateAutoConf.py +1
-0@@ -119,6 +120,18 @@ class CreateAutoConf(ShellCommand): else: adtrepo_url=os.environ.get("ADTREPO_URL") fout=fout+'ADTREPO = "' + adtrepo_url + '/' + self.getProperty("distroversion") + '"\n' + if self.getProperty("branch_poky") != "danny" and self.initmgr:
Here we will probably want to do:
if "danny" not in self.getProperty("branch_poky")....
That covers cases where we have "danny-next" or some other issue. I'm sure I do what you do above, and if so, that should be fixed.
Quoted 15 lines
lib/python2.7/site-packages/autobuilder/buildsteps/CreateAutoConf.py +11
-1+ # we don't neet to test sysvinit only, because that's the default + if self.initmgr == "systemd": + fout = fout + 'DISTRO_FEATURES_append = " systemd"\n' + fout = fout + 'VIRTUAL-RUNTIME_init_manager = "systemd"\n' + fout = fout + 'DISTRO_FEATURES_BACKFILL_CONSIDERED = "sysvinit"\n' + if self.initmgr == "systemd sysvinit": + fout = fout + 'DISTRO_FEATURES_append = " systemd"\n' + fout = fout + 'VIRTUAL-RUNTIME_init_manager = "systemd"\n' + if self.initmgr == "sysvinit systemd": + fout = fout + 'DISTRO_FEATURES_append = " systemd"\n' + fout = fout + 'VIRTUAL-RUNTIME_init_manager = "sysvinit"\n' if self.atextappend: fout = fout + self.atextappend --
Quoted 2 lines
1.8.1.4
--
Elizabeth Flanagan
Yocto Project
Build and Release
Yes to these with the one fixed I suggest and a rebase. Nice work.
-b
Quoted 42 lines · Stefan Stanacar
Hello Beth,
Would you mind taking a look on these changes? Obviously they are meant post-dylan release, but I sent them earlier so you can have time to review and request changes.
It would be great to have them in 1.5 and I plan on adding even more builds (especially for multilib images and ipk images which proved tricky with postinstalls and systemd).Thanks,
StefanThe following changes since commit
0178163efc6670c9acb982dd579671396cb24290:
PublishLayerTarballs.py: bad redirect. (2013-04-22 22:56:58 -0700)are available in the git repository at:
git://git.yoctoproject.org/poky-contribstefans/yab0
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=stefans/yab0Stefan Stanacar (6):
nightly-qa-extras.conf: add new buildset configCreateAutoConf.py: add initmgr optionbuildset-config/nightly-qa-systemd.conf: add a new target for systemd
images
nightly.conf, yoctoAB.conf: enable the new buildsetsbuildsteps/RunSanityTests.py: add suport for custom stepsCreateAutoConf.py: add option for package typesbuildset-config/nightly-qa-extras.conf | 16 ++++++++++++++++ buildset-config/nightly-qa-systemd.conf | 20 ++++++++++++++++++++ buildset-config/nightly.conf | 3 ++- buildset-config/yoctoAB.conf | 3 ++- .../autobuilder/buildsteps/CreateAutoConf.py | 20 +++++++++++++++++++- .../autobuilder/buildsteps/RunSanityTests.py | 3 +++ 6 files changed, 62 insertions(+), 3 deletions(-) create mode 100644 buildset-config/nightly-qa-extras.conf create mode 100644 buildset-config/nightly-qa-systemd.conf--
1.8.1.4
Signature · 3 lines
Elizabeth Flanagan Yocto Project Build and Release