package_manager: support for signed DEB package feeds
[PATCH v4 1/2] apt: add apt selftest to test signed package feeds [PATCH v4 2/2] package_manager: fix missing dependency on gnupg when
Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.
Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign deb package feeds" (already in master) enable signed deb package feeds. When called fromoe-selftest -r runtime_test.TestImage.test_testimage_apt this patch adds a runtime test for apt derived from the test_testimage_dnf test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.
Changes in V4:
- Add fix to make gnupg-native a dependency else hosttools is used and oe-selftest -r runtime_test.TestImage.test_testimage_apt fails on Ubuntu 16.04 used on the autobuilder (Alexandre Belloni)
Changes in V3:
- When called from bitbake core-image-sato -c testimage package feed is unsigned. Auto-detect this case and behave as before (Richard Purdie)
Changes in V2:
- Added runtime test for signed deb package feeds (Richard Purdie)
PATCH v4 1/2 apt: add apt selftest to test signed package feeds
From: Ferry Toth <[email protected]>
Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.
Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign DEB package feeds" enable signed DEB package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.
--- meta/lib/oeqa/runtime/cases/apt.py | 38 ++++++++++++++++---- meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++ 2 files changed, 69 insertions(+), 7 deletions(-)
meta/lib/oeqa/runtime/cases/apt.py +31 -7
@@ -21,7 +21,7 @@ class AptRepoTest(AptTest): @classmethod def setUpClass(cls): - service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all') + service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '') cls.repo_server = HTTPService(service_repo, '0.0.0.0', port=cls.tc.target.server_port, logger=cls.tc.logger)
@@ -34,20 +34,44 @@ class AptRepoTest(AptTest): def setup_source_config_for_package_install(self): apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port) apt_get_sourceslist_dir = '/etc/apt/' - self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server)) + self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s/all ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server)) + + def setup_source_config_for_package_install_signed(self): + apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port) + apt_get_sourceslist_dir = '/etc/apt/' + self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server)) def cleanup_source_config_for_package_install(self): apt_get_sourceslist_dir = '/etc/apt/' self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir)) + def cleanup_source_config_for_package_install_signed(self): + apt_get_sourceslist_dir = '/etc/apt/' + self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir)) + + def setup_key(self): + # the key is found on the target /etc/pki/packagefeed-gpg/ + # named PACKAGEFEED-GPG-KEY-poky-branch + self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg')) + @skipIfNotFeature('package-management', 'Test requires package-management to be in IMAGE_FEATURES') @skipIfNotDataVar('IMAGE_PKGTYPE', 'deb', 'DEB is not the primary package manager') @OEHasPackage(['apt']) def test_apt_install_from_repo(self): - self.setup_source_config_for_package_install() - self.pkg('update') - self.pkg('remove --yes run-postinsts-dev') - self.pkg('install --yes --allow-unauthenticated run-postinsts-dev') - self.cleanup_source_config_for_package_install() + if not self.tc.td.get('PACKAGE_FEED_GPG_NAME'): + self.setup_source_config_for_package_install() + self.pkg('update') + self.pkg('remove --yes run-postinsts-dev') + self.pkg('install --yes --allow-unauthenticated run-postinsts-dev') + self.cleanup_source_config_for_package_install() + else: + # when we are here a key has been set to sign the package feed and + # public key and gnupg installed on the image by test_testimage_apt + self.setup_source_config_for_package_install_signed() + self.setup_key() + self.pkg('update') + self.pkg('install --yes run-postinsts-dev') + self.pkg('remove --yes run-postinsts-dev') + self.cleanup_source_config_for_package_install_signed()
meta/lib/oeqa/selftest/cases/runtime_test.py +38 -0
@@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase): bitbake('core-image-full-cmdline socat') bitbake('-c testimage core-image-full-cmdline') + def test_testimage_apt(self): + """ + Summary: Check package feeds functionality for apt + Expected: 1. Check that remote package feeds can be accessed + Product: oe-core + Author: Ferry Toth <[email protected]> + """ + if get_bb_var('DISTRO') == 'poky-tiny': + self.skipTest('core-image-full-cmdline not buildable for poky-tiny') + + features = 'INHERIT += "testimage"\n' + features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n' + # We don't yet know what the server ip and port will be - they will be patched + # in at the start of the on-image test + features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n' + features += 'EXTRA_IMAGE_FEATURES += "package-management"\n' + features += 'PACKAGE_CLASSES = "package_deb"\n' + # We need gnupg on the target to install keys + features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n' + + bitbake('gnupg-native -c addto_recipe_sysroot') + + # Enable package feed signing + self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-") + self.track_for_cleanup(self.gpg_home) + signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing') + runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True) + features += 'INHERIT += "sign_package_feed"\n' + features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n' + features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase') + features += 'GPG_PATH = "%s"\n' % self.gpg_home + features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home + self.write_config(features) + + # Build core-image-sato and testimage + bitbake('core-image-full-cmdline socat') + bitbake('-c testimage core-image-full-cmdline') + def test_testimage_virgl_gtk_sdl(self): """ Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontends
Signature · 1 line
2.32.0
PATCH v4 2/2 package_manager: fix missing dependency on gnupg when signing deb package feeds
In reply to Ferry TothFrom: Ferry Toth <[email protected]>
When signing the deb package feed gpg tools are a soft requirement. If gnupg-native is not declared a dependancy the version from hosttools is used. Unfortunately the gpg-agent version from Ubuntu 16.04 on the autobuilders is incompatible with the package_index task and fails during oe-selftest. Fix by making gnupg-native a dependency.
--- meta/classes/sign_package_feed.bbclass | 1 + 1 file changed, 1 insertion(+)
meta/classes/sign_package_feed.bbclass +1 -0
@@ -27,6 +27,7 @@ inherit sanity PACKAGE_FEED_SIGN = '1' PACKAGE_FEED_GPG_BACKEND ?= 'local' PACKAGE_FEED_GPG_SIGNATURE_TYPE ?= 'ASC' +PACKAGEINDEXDEPS += "gnupg-native:do_populate_sysroot" # Make feed signing key to be present in rootfs FEATURE_PACKAGES_package-management:append = " signing-keys-packagefeed"
Signature · 1 line
2.32.0
Quoted 19 lines · Ferry Toth
[PATCH v4 1/2] apt: add apt selftest to test signed package feeds [PATCH v4 2/2]
package_manager: fix missing dependency on gnupg whenSince Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.
Patches "gpg-sign: Add parameters to gpg signature function" and "
package_manager: sign deb package feeds" (already in master) enable signed deb package feeds. When called fromoe-selftest -r runtime_test.TestImage.test_testimage_aptthis patch adds a runtime test for apt derived from thetest_testimage_dnftest. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.Changes in V4:
- Add fix to make gnupg-native a dependency else hosttools is used andoe-selftest -r runtime_test.TestImage.test_testimage_aptfails on Ubuntu 16.04 used on the autobuilder (Alexandre Belloni)
Quoted 6 lines
Changes in V3:
- When called frombitbake core-image-sato -c testimagepackage feed is unsigned. Auto-detect this case and behave as before (Richard Purdie)Changes in V2:
- Added runtime test for signed deb package feeds (Richard Purdie)
This has now merged, thanks for working through the details with this! The test should allow the functionality to stay working and is extremely useful/helpful.
Cheers,
Richard
Hi,
Op 19-04-2022 om 15:21 schreef Richard Purdie:
Quoted 24 lines · Ferry Toth
Quoted 23 lines
[PATCH v4 1/2] apt: add apt selftest to test signed package feeds [PATCH v4 2/2]
package_manager: fix missing dependency on gnupg whenSince Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.
Patches "gpg-sign: Add parameters to gpg signature function" and "
package_manager: sign deb package feeds" (already in master) enable signed deb package feeds. When called fromoe-selftest -r runtime_test.TestImage.test_testimage_aptthis patch adds a runtime test for apt derived from thetest_testimage_dnftest. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.Changes in V4:
- Add fix to make gnupg-native a dependency else hosttools is used andoe-selftest -r runtime_test.TestImage.test_testimage_aptfails on Ubuntu 16.04 used on the autobuilder (Alexandre Belloni)Changes in V3:
- When called frombitbake core-image-sato -c testimagepackage feed is unsigned. Auto-detect this case and behave as before (Richard Purdie)
Quoted 5 lines
Quoted 2 lines
Changes in V2:
- Added runtime test for signed deb package feeds (Richard Purdie)This has now merged, thanks for working through the details with this! The test should allow the functionality to stay working and is extremely useful/helpful.
I watched it go through the CI bots and I must say I'm impressed with the process. Thanks for guiding me through this and merging.
Quoted 4 lines
Cheers,
Richard