Skip to contentSkip to conversation

package_manager: support for signed DEB package feeds

PATCH New Ferry Toth 2 participants 5 messages Apr 13, 2022 – Apr 19, 2022

lore
Share & tools
lore

Ferry Toth

author
#0 lore raw

[PATCH v4 1/2] apt: add apt selftest to test signed package feeds [PATCH v4 2/2] package_manager: fix missing dependency on gnupg when

Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.

Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign deb package feeds" (already in master) enable signed deb package feeds. When called from
oe-selftest -r runtime_test.TestImage.test_testimage_apt this patch adds a runtime test for apt derived from the test_testimage_dnf test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.

Changes in V4:
- Add fix to make gnupg-native a dependency else hosttools is used and oe-selftest -r runtime_test.TestImage.test_testimage_apt fails on Ubuntu 16.04 used on the autobuilder (Alexandre Belloni)

Changes in V3:
- When called from bitbake core-image-sato -c testimage package feed is unsigned. Auto-detect this case and behave as before (Richard Purdie)

Changes in V2:
- Added runtime test for signed deb package feeds (Richard Purdie)

author
#1 lore raw

PATCH v4 1/2 apt: add apt selftest to test signed package feeds

From: Ferry Toth <[email protected]>

Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.

Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign DEB package feeds" enable signed DEB package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.

---
 meta/lib/oeqa/runtime/cases/apt.py           | 38 ++++++++++++++++----
 meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++
 2 files changed, 69 insertions(+), 7 deletions(-)
meta/lib/oeqa/runtime/cases/apt.py +31 -7
index 53745df93f..574a34f148 100644
@@ -21,7 +21,7 @@ class AptRepoTest(AptTest):
 
     @classmethod
     def setUpClass(cls):
-        service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all')
+        service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '')
         cls.repo_server = HTTPService(service_repo,
                                       '0.0.0.0', port=cls.tc.target.server_port,
                                       logger=cls.tc.logger)
@@ -34,20 +34,44 @@ class AptRepoTest(AptTest):
     def setup_source_config_for_package_install(self):
         apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port)
         apt_get_sourceslist_dir = '/etc/apt/'
-        self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
+        self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s/all ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server))
+
+    def setup_source_config_for_package_install_signed(self):
+        apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port)
+        apt_get_sourceslist_dir = '/etc/apt/'
+        self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server))
 
     def cleanup_source_config_for_package_install(self):
         apt_get_sourceslist_dir = '/etc/apt/'
         self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir))
 
+    def cleanup_source_config_for_package_install_signed(self):
+        apt_get_sourceslist_dir = '/etc/apt/'
+        self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir))
+
+    def setup_key(self):
+        # the key is found on the target /etc/pki/packagefeed-gpg/
+        # named PACKAGEFEED-GPG-KEY-poky-branch
+        self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg'))
+
     @skipIfNotFeature('package-management',
                       'Test requires package-management to be in IMAGE_FEATURES')
     @skipIfNotDataVar('IMAGE_PKGTYPE', 'deb',
                       'DEB is not the primary package manager')
     @OEHasPackage(['apt'])
     def test_apt_install_from_repo(self):
-        self.setup_source_config_for_package_install()
-        self.pkg('update')
-        self.pkg('remove --yes run-postinsts-dev')
-        self.pkg('install --yes --allow-unauthenticated run-postinsts-dev')
-        self.cleanup_source_config_for_package_install()
+        if not self.tc.td.get('PACKAGE_FEED_GPG_NAME'):
+            self.setup_source_config_for_package_install()
+            self.pkg('update')
+            self.pkg('remove --yes run-postinsts-dev')
+            self.pkg('install --yes --allow-unauthenticated run-postinsts-dev')
+            self.cleanup_source_config_for_package_install()
+        else:
+            # when we are here a key has been set to sign the package feed and
+            # public key and gnupg installed on the image by test_testimage_apt
+            self.setup_source_config_for_package_install_signed()
+            self.setup_key()
+            self.pkg('update')
+            self.pkg('install --yes run-postinsts-dev')
+            self.pkg('remove --yes run-postinsts-dev')
+            self.cleanup_source_config_for_package_install_signed()
meta/lib/oeqa/selftest/cases/runtime_test.py +38 -0
index 2ad89490fc..3ece617cb0 100644
@@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase):
         bitbake('core-image-full-cmdline socat')
         bitbake('-c testimage core-image-full-cmdline')
 
+    def test_testimage_apt(self):
+        """
+        Summary: Check package feeds functionality for apt
+        Expected: 1. Check that remote package feeds can be accessed
+        Product: oe-core
+        Author: Ferry Toth <[email protected]>
+        """
+        if get_bb_var('DISTRO') == 'poky-tiny':
+            self.skipTest('core-image-full-cmdline not buildable for poky-tiny')
+
+        features = 'INHERIT += "testimage"\n'
+        features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n'
+        # We don't yet know what the server ip and port will be - they will be patched
+        # in at the start of the on-image test
+        features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n'
+        features += 'EXTRA_IMAGE_FEATURES += "package-management"\n'
+        features += 'PACKAGE_CLASSES = "package_deb"\n'
+        # We need  gnupg on the target to install keys
+        features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n'
+
+        bitbake('gnupg-native -c addto_recipe_sysroot')
+
+        # Enable package feed signing
+        self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-")
+        self.track_for_cleanup(self.gpg_home)
+        signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing')
+        runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True)
+        features += 'INHERIT += "sign_package_feed"\n'
+        features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n'
+        features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase')
+        features += 'GPG_PATH = "%s"\n' % self.gpg_home
+        features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home
+        self.write_config(features)
+
+        # Build core-image-sato and testimage
+        bitbake('core-image-full-cmdline socat')
+        bitbake('-c testimage core-image-full-cmdline')
+
     def test_testimage_virgl_gtk_sdl(self):
         """
         Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontends
Signature · 1 line
2.32.0
author
#2 lore raw

PATCH v4 2/2 package_manager: fix missing dependency on gnupg when signing deb package feeds

In reply to Ferry Toth

From: Ferry Toth <[email protected]>

When signing the deb package feed gpg tools are a soft requirement. If gnupg-native is not declared a dependancy the version from hosttools is used. Unfortunately the gpg-agent version from Ubuntu 16.04 on the autobuilders is incompatible with the package_index task and fails during oe-selftest. Fix by making gnupg-native a dependency.

Fixes 0b4231b5 "package_manager: sign DEB package feeds"
---
 meta/classes/sign_package_feed.bbclass | 1 +
 1 file changed, 1 insertion(+)
meta/classes/sign_package_feed.bbclass +1 -0
index 16bcd147aa..f1504c2225 100644
@@ -27,6 +27,7 @@ inherit sanity
 PACKAGE_FEED_SIGN = '1'
 PACKAGE_FEED_GPG_BACKEND ?= 'local'
 PACKAGE_FEED_GPG_SIGNATURE_TYPE ?= 'ASC'
+PACKAGEINDEXDEPS += "gnupg-native:do_populate_sysroot"
 
 # Make feed signing key to be present in rootfs
 FEATURE_PACKAGES_package-management:append = " signing-keys-packagefeed"
Signature · 1 line
2.32.0
#3 lore raw
In reply to Ferry Toth
Quoted 19 lines · Ferry Toth

[PATCH v4 1/2] apt: add apt selftest to test signed package feeds [PATCH v4 2/2] package_manager: fix missing dependency on gnupg when

Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.

Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign deb package feeds" (already in master) enable signed deb package feeds. When called from
oe-selftest -r runtime_test.TestImage.test_testimage_apt this patch adds a runtime test for apt derived from the test_testimage_dnf test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.

Changes in V4:
- Add fix to make gnupg-native a dependency else hosttools is used and oe-selftest -r runtime_test.TestImage.test_testimage_apt fails on Ubuntu 16.04 used on the autobuilder (Alexandre Belloni)

Quoted 6 lines

Changes in V3:
- When called from bitbake core-image-sato -c testimage package feed is unsigned. Auto-detect this case and behave as before (Richard Purdie)

Changes in V2:
- Added runtime test for signed deb package feeds (Richard Purdie)

This has now merged, thanks for working through the details with this! The test should allow the functionality to stay working and is extremely useful/helpful.

Cheers,

Richard

author
#4 lore raw

Hi,

Op 19-04-2022 om 15:21 schreef Richard Purdie:

Quoted 24 lines · Ferry Toth
Quoted 23 lines

[PATCH v4 1/2] apt: add apt selftest to test signed package feeds [PATCH v4 2/2] package_manager: fix missing dependency on gnupg when

Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.

Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign deb package feeds" (already in master) enable signed deb package feeds. When called from
oe-selftest -r runtime_test.TestImage.test_testimage_apt this patch adds a runtime test for apt derived from the test_testimage_dnf test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.

Changes in V4:
- Add fix to make gnupg-native a dependency else hosttools is used and oe-selftest -r runtime_test.TestImage.test_testimage_apt fails on Ubuntu 16.04 used on the autobuilder (Alexandre Belloni)

Changes in V3:
- When called from bitbake core-image-sato -c testimage package feed is unsigned. Auto-detect this case and behave as before (Richard Purdie)

Quoted 5 lines
Quoted 2 lines

Changes in V2:
- Added runtime test for signed deb package feeds (Richard Purdie)

This has now merged, thanks for working through the details with this! The test should allow the functionality to stay working and is extremely useful/helpful.

I watched it go through the CI bots and I must say I'm impressed with the process. Thanks for guiding me through this and merging.

Quoted 4 lines

Cheers,

Richard