oeqa/runtime/cases: new image_upgrade test
RFC PATCH 1/1 oeqa/runtime/cases: new image_upgrade test
From: Michael Opdenacker <[email protected]>
New oe-selftest and associated "testimage" test to check that generated package feeds can be used to update the latest image built by the Yocto Project autobuilder.
Currently, only the "core-image-full-cmdline" image with IPK packages and the "poky-altcfg" distro is supported.
Test it by running:
oe-selftest -r image_upgrade
---
Tested on the latest master against yocto-5.0_M3
Interested in your feedback about which settings (such as IMAGE) can be hardcoded in the scripts or provided by the calling environment.
--- .../lib/oeqa/runtime/cases/opkg_sysupgrade.py | 68 ++++++++++ meta/lib/oeqa/selftest/cases/image_upgrade.py | 124 ++++++++++++++++++ 2 files changed, 192 insertions(+) create mode 100644 meta/lib/oeqa/runtime/cases/opkg_sysupgrade.py create mode 100755 meta/lib/oeqa/selftest/cases/image_upgrade.py
meta/lib/oeqa/runtime/cases/opkg_sysupgrade.py new file +68 -0
@@ -0,0 +1,68 @@ +# +# Copyright OpenEmbedded Contributors +# +# Test that generated ipk packages can be used to upgrade +# an older image version. +# +# This is done by the meta/lib/oeqa/selftest/cases/image_upgrade.py oe-selftest +# replacing the newly generated image by an older image +# generated by the Yocto Project autobuilder. +# +# Here, we replace the package feeds in our image by our own +# +# This test is not meant to be used as a regular "testimage" test +# run on the fresh image. +# +# SPDX-License-Identifier: MIT +# + +import os +from oeqa.utils.httpserver import HTTPService +from oeqa.runtime.case import OERuntimeTestCase +from oeqa.core.decorator.data import skipIfNotDataVar, skipIfNotFeature, skipIfFeature +from oeqa.runtime.decorator.package import OEHasPackage + +class OpkgSysUpgradeTest(OERuntimeTestCase): + + def pkg(self, command, expected = 0): + command = 'opkg %s' % command + status, output = self.target.run(command, 1500) + message = os.linesep.join([command, output]) + self.assertEqual(status, expected, message) + return output + +class OpkgRepoTest(OpkgSysUpgradeTest): + + @classmethod + def setUp(cls): + service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_IPK']) + cls.repo_server = HTTPService(service_repo, + '0.0.0.0', port=cls.tc.target.server_port, + logger=cls.tc.logger) + cls.repo_server.start() + + @classmethod + def tearDown(cls): + cls.repo_server.stop() + + def setup_source_config_for_package_install(self): + source_server = 'http://%s:%s' % (self.tc.target.server_ip, self.repo_server.port) + sourceslist_dir = '/etc/opkg' + pkgarch = self.tc.td["TUNE_PKGARCH"] + machinedir = self.tc.td["MACHINE"].replace("-", "_") + self.target.run('cd %s; echo src/gz all %s/all > base-feeds.conf' % (sourceslist_dir, source_server)) + self.target.run('cd %s; echo src/gz %s %s/%s >> base-feeds.conf' % (sourceslist_dir, pkgarch, source_server, pkgarch)) + self.target.run('cd %s; echo src/gz %s %s/%s >> base-feeds.conf' % (sourceslist_dir, machinedir, source_server, machinedir)) + + @skipIfNotFeature('package-management', + 'Test requires package-management to be in IMAGE_FEATURES') + @skipIfNotDataVar('IMAGE_PKGTYPE', 'ipk', + 'IPK is not the primary package manager') + @skipIfFeature('read-only-rootfs', + 'Test does not work with read-only-rootfs in IMAGE_FEATURES') + @OEHasPackage(['opkg']) + def test_opkg_system_upgrade_from_repo(self): + self.setup_source_config_for_package_install() + self.pkg('update') + self.pkg('upgrade') +
meta/lib/oeqa/selftest/cases/image_upgrade.py new file +124 -0
124 lines not on this page · raw message
Signature · 1 line
2.34.1
Hi Michael,
At least at a first read and without running it, this does look like a reasonable direction. I suspect that anyone else looking at this would have a lot of questions about why we'd do it this way but given the various constraints, it does make sense to me.
What we do need to think about is how someone else would reuse this as currently it is very poky specific. Our aim is to make it easy for others to use too. With that in mind:
* We probably want to "tag" this test with something so we can exclude it from the normal oe-selftest runs on the autobuilder and allow it to run on a per machine basis. There are other oe-selftests we already do this with (like toolchain testing or machine specific environment file tests).
* The configuration about what to test probably needs to come from the distro (i.e. which DISTRO/MACHINE/image combinations).
* We probably need to parameterise it so that a list of images can be tested rather than just a single one. I did wonder if we could have it dynamically add tests for each image configured.
* We don't want to test on all MACHINE (e.g. qemumips and qemuppc are not going to be included).
* We need to find a better way to share the code with autobuilder- helper, I don't like duplicating code.
* The image url code is also highly poky specific. That probably needs to come from the poky repository alongside the configuration.
Does that all make sense?
Cheers,
Richard
Hi Richard,
Many thanks for your feedback.
I started by implementing Alex Kanavin's suggestions, which are a little more specific to get started ;-)
Quoted 24 lines · Richard Purdie
Hi Michael,
At least at a first read and without running it, this does look like a reasonable direction. I suspect that anyone else looking at this would have a lot of questions about why we'd do it this way but given the various constraints, it does make sense to me.
What we do need to think about is how someone else would reuse this as currently it is very poky specific. Our aim is to make it easy for others to use too. With that in mind:
* We probably want to "tag" this test with something so we can exclude it from the normal oe-selftest runs on the autobuilder and allow it to run on a per machine basis. There are other oe-selftests we already do this with (like toolchain testing or machine specific environment file tests).
* The configuration about what to test probably needs to come from the distro (i.e. which DISTRO/MACHINE/image combinations).
* We probably need to parameterise it so that a list of images can be tested rather than just a single one. I did wonder if we could have it dynamically add tests for each image configured.
Quoted 5 lines
* We don't want to test on all MACHINE (e.g. qemumips and qemuppc are not going to be included).
* We need to find a better way to share the code with autobuilder- helper, I don't like duplicating code.
Is there example code somewhere doing something similar, i.e. importing code from an external repository?
Quoted 3 lines
* The image url code is also highly poky specific. That probably needs to come from the poky repository alongside the configuration.
I'm going to submit a new version that doesn't assume any specific URL scheme. More soon.
Thanks,
Michael.
Signature · 3 lines
Michael Opdenacker, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Quoted 25 lines · Michael Opdenacker
Quoted 24 lines · Richard Purdie
Hi Michael,
At least at a first read and without running it, this does look like a reasonable direction. I suspect that anyone else looking at this would have a lot of questions about why we'd do it this way but given the various constraints, it does make sense to me.
What we do need to think about is how someone else would reuse this as currently it is very poky specific. Our aim is to make it easy for others to use too. With that in mind:
* We probably want to "tag" this test with something so we can exclude it from the normal oe-selftest runs on the autobuilder and allow it to run on a per machine basis. There are other oe-selftests we already do this with (like toolchain testing or machine specific environment file tests).
* The configuration about what to test probably needs to come from the distro (i.e. which DISTRO/MACHINE/image combinations).
* We probably need to parameterise it so that a list of images can be tested rather than just a single one. I did wonder if we could have it dynamically add tests for each image configured.
Quoted 8 lines
Quoted 5 lines
* We don't want to test on all MACHINE (e.g. qemumips and qemuppc are not going to be included).
* We need to find a better way to share the code with autobuilder- helper, I don't like duplicating code.
Is there example code somewhere doing something similar, i.e. importing code from an external repository?
What needs to happen is the code needs to be moved into meta/lib/oe, or probably better, meta-poky/lib/oe in meta-yocto and then autobuilder- helper needs to understand how to access it from there.
That is probably beyond what we have time for with the current work :(
Quoted 6 lines
Quoted 2 lines
* The image url code is also highly poky specific. That probably needs to come from the poky repository alongside the configuration.
I'm going to submit a new version that doesn't assume any specific URL scheme.
More soon.
Some assumptions about the url format will have to be made. I still maintain this code should be in meta-yocto since it is poky specific but should call code with only the configuration coming from poky/meta- yocto.
Cheers,
Richard
Quoted 1 line · Michael Opdenacker via lists.openembedded.org
+++
b/meta/lib/oeqa/runtime/cases/opkg_sysupgrade.py
This should perhaps go to meta-selftest/lib/oeqa/runtime/cases/ as it needs special setup via selftest, and not useful in standalone '-c testimage' runs.
Quoted 34 lines
meta/lib/oeqa/selftest/cases/image_upgrade.py new file +34
-0+ def test_image_upgrade(self): + """ + Summary: Test that generated ipk packages can + be used to upgrade an older image version. + This is done by generating an image but then replacing it + by an older image shared by the Yocto Project autobuilder. + We then run QEMU on the old image and replace the original + original package feeds by our own. + """ + + image = 'core-image-full-cmdline' + machine = get_bb_var("MACHINE") + + features = 'DISTRO = "poky-altcfg"\n' + features += 'EXTRA_IMAGE_FEATURES += "package-management"\n' + features += 'PACKAGE_CLASSES = "package_ipk"\n' + features += 'IMAGE_CLASSES += "testimage"\n' + features += 'TEST_SUITES="opkg_sysupgrade"\n' + self.write_config(features) + + # Need to build a full image to build the .json file needed by QEMU. + # Therefore, it is not sufficient to run only "package_write_ipk" for the image. + + self.logger.info("Generating '%s' and package index for latest commit in this branch..." % image) + bitbake(image) + bitbake('package-index') + + # Download previously generated image + + image_file = '%s-%s.rootfs.ext4' % (image, machine) + image_path = '%s/tmp/deploy/images/%s/%s' % (self.builddir, machine, image_file) + machine_variant = '-alt' + + os.remove(image_path)
meta/lib/oeqa/selftest/cases/image_upgrade.py line 124
Quoted 10 lines
meta/lib/oeqa/selftest/cases/image_upgrade.py new file +10
-0+ image_url = get_latest_image_url(machine, machine_variant, image_file) + self.logger.info("Downloading image: %s..." % image_url) + cmd = 'wget -O %s %s' % (image_path, image_url) + result = runCmd(cmd) + self.assertEqual(0, result.status, cmd + ' returned a non 0 status: %s' % result.output) + + # Now run the upgrade tests on the old image + + self.logger.info("Running upgrade tests on the downloaded image, using the package feeds generated here...") + bitbake(image + ' -c testimage')
RP perhaps already pointed out this, but this mashes together parameters and implementation. Please split the implementation into 'non-opinionated' functions that take everything that can change via arguments, and have a short and sweet top level test definition that does something like this:
def test_poky_altcfg_cmdline_update(self):
config = 'DISTRO=..., etc'
imagelocation = find_yocto_image(...)
run_update_test(config, imagelocation, ...)
As an example, CDN sstate tests are arranged similarly:
https://git.yoctoproject.org/poky/tree/meta/lib/oeqa/selftest/cases/sstatetests.py#n920
Then we can further improve this by defining those parameters somewhere external, that's a separate decision.
Alex