package_manager: support for signed DEB package feeds
[PATCH v3 1/1] apt: add apt selftest to test signed package feeds
Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.
Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign deb package feeds" (already in master) enable signed deb package feeds. When called fromoe-selftest -r runtime_test.TestImage.test_testimage_apt this patch adds a runtime test for apt derived from the test_testimage_dnf test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.
Changes in V3:
- When called from bitbake core-image-sato -c testimage package feed is unsigned. Auto-detect this case and behave as before (Richard Purdie)
Changes in V2:
- Added runtime test for signed deb package feeds (Richard Purdie)
PATCH v3 1/1 apt: add apt selftest to test signed package feeds
From: Ferry Toth <[email protected]>
Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.
Patches "gpg-sign: Add parameters to gpg signature function" and "package_manager: sign DEB package feeds" enable signed DEB package feeds. This patch adds a runtime test for apt derived from the test_testimage_dnf test. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.
--- meta/lib/oeqa/runtime/cases/apt.py | 38 ++++++++++++++++---- meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++ 2 files changed, 69 insertions(+), 7 deletions(-)
meta/lib/oeqa/runtime/cases/apt.py +31 -7
@@ -21,7 +21,7 @@ class AptRepoTest(AptTest): @classmethod def setUpClass(cls): - service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all') + service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '') cls.repo_server = HTTPService(service_repo, '0.0.0.0', port=cls.tc.target.server_port, logger=cls.tc.logger)
@@ -34,20 +34,44 @@ class AptRepoTest(AptTest): def setup_source_config_for_package_install(self): apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port) apt_get_sourceslist_dir = '/etc/apt/' - self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server)) + self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s/all ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server)) + + def setup_source_config_for_package_install_signed(self): + apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port) + apt_get_sourceslist_dir = '/etc/apt/' + self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server)) def cleanup_source_config_for_package_install(self): apt_get_sourceslist_dir = '/etc/apt/' self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir)) + def cleanup_source_config_for_package_install_signed(self): + apt_get_sourceslist_dir = '/etc/apt/' + self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir)) + + def setup_key(self): + # the key is found on the target /etc/pki/packagefeed-gpg/ + # named PACKAGEFEED-GPG-KEY-poky-branch + self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg')) + @skipIfNotFeature('package-management', 'Test requires package-management to be in IMAGE_FEATURES') @skipIfNotDataVar('IMAGE_PKGTYPE', 'deb', 'DEB is not the primary package manager') @OEHasPackage(['apt']) def test_apt_install_from_repo(self): - self.setup_source_config_for_package_install() - self.pkg('update') - self.pkg('remove --yes run-postinsts-dev') - self.pkg('install --yes --allow-unauthenticated run-postinsts-dev') - self.cleanup_source_config_for_package_install() + if not self.tc.td.get('PACKAGE_FEED_GPG_NAME'): + self.setup_source_config_for_package_install() + self.pkg('update') + self.pkg('remove --yes run-postinsts-dev') + self.pkg('install --yes --allow-unauthenticated run-postinsts-dev') + self.cleanup_source_config_for_package_install() + else: + # when we are here a key has been set to sign the package feed and + # public key and gnupg installed on the image by test_testimage_apt + self.setup_source_config_for_package_install_signed() + self.setup_key() + self.pkg('update') + self.pkg('install --yes run-postinsts-dev') + self.pkg('remove --yes run-postinsts-dev') + self.cleanup_source_config_for_package_install_signed()
meta/lib/oeqa/selftest/cases/runtime_test.py +38 -0
@@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase): bitbake('core-image-full-cmdline socat') bitbake('-c testimage core-image-full-cmdline') + def test_testimage_apt(self): + """ + Summary: Check package feeds functionality for apt + Expected: 1. Check that remote package feeds can be accessed + Product: oe-core + Author: Ferry Toth <[email protected]> + """ + if get_bb_var('DISTRO') == 'poky-tiny': + self.skipTest('core-image-full-cmdline not buildable for poky-tiny') + + features = 'INHERIT += "testimage"\n' + features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n' + # We don't yet know what the server ip and port will be - they will be patched + # in at the start of the on-image test + features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n' + features += 'EXTRA_IMAGE_FEATURES += "package-management"\n' + features += 'PACKAGE_CLASSES = "package_deb"\n' + # We need gnupg on the target to install keys + features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n' + + bitbake('gnupg-native -c addto_recipe_sysroot') + + # Enable package feed signing + self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-") + self.track_for_cleanup(self.gpg_home) + signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing') + runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True) + features += 'INHERIT += "sign_package_feed"\n' + features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n' + features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase') + features += 'GPG_PATH = "%s"\n' % self.gpg_home + features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home + self.write_config(features) + + # Build core-image-sato and testimage + bitbake('core-image-full-cmdline socat') + bitbake('-c testimage core-image-full-cmdline') + def test_testimage_virgl_gtk_sdl(self): """ Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontends
Signature · 1 line
2.32.0
Re: [OE-core] [PATCH v3 1/1] apt: add apt selftest to test signed package feeds
Hello,
Quoted 11 lines · Ferry Toth
From: Ferry Toth <[email protected]>
Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.
Patches "gpg-sign: Add parameters to gpg signature function" and "
package_manager: sign DEB package feeds" enable signed DEB package feeds. This patch adds a runtime test for apt derived from thetest_testimage_dnftest. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.
This went through the autobuilders and it seems this still fails:
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/3437/steps/15/logs/stdio
ERROR: package-index-1.0-r0 do_package_index: Could not get gpg version: Command '['/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg', '--agent-program=/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg-agent|--auto-expand-secmem', '--version', '--no-permission-warning']' returned non-zero exit status 2.
ERROR: Logfile of failure stored in: /home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/work/core2-64-poky-linux/package-index/1.0-r0/temp/log.do_package_index.53841
NOTE: recipe package-index-1.0-r0: task do_package_index: Failed
ERROR: Task (/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/recipes-core/meta/package-index.bb:do_package_index) failed with exit code '1'
This was ubuntu 16.04 so maybe gpg on the distro is too old (1.4.20) but I'm not sure as I think you are using gnupg-native.
Quoted 126 lines
Signed-off-by Ferry Toth <[email protected]>--- meta/lib/oeqa/runtime/cases/apt.py | 38 ++++++++++++++++---- meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++ 2 files changed, 69 insertions(+), 7 deletions(-)meta/lib/oeqa/runtime/cases/apt.py +31
-7@@ -21,7 +21,7 @@ class AptRepoTest(AptTest): @classmethod def setUpClass(cls): - service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all') + service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '') cls.repo_server = HTTPService(service_repo, '0.0.0.0', port=cls.tc.target.server_port, logger=cls.tc.logger)@@ -34,20 +34,44 @@ class AptRepoTest(AptTest): def setup_source_config_for_package_install(self): apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port) apt_get_sourceslist_dir = '/etc/apt/' - self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server)) + self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s/all ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server)) + + def setup_source_config_for_package_install_signed(self): + apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port) + apt_get_sourceslist_dir = '/etc/apt/' + self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server)) def cleanup_source_config_for_package_install(self): apt_get_sourceslist_dir = '/etc/apt/' self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir)) + def cleanup_source_config_for_package_install_signed(self): + apt_get_sourceslist_dir = '/etc/apt/' + self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir)) + + def setup_key(self): + # the key is found on the target /etc/pki/packagefeed-gpg/ + # named PACKAGEFEED-GPG-KEY-poky-branch + self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg')) + @skipIfNotFeature('package-management', 'Test requires package-management to be in IMAGE_FEATURES') @skipIfNotDataVar('IMAGE_PKGTYPE', 'deb', 'DEB is not the primary package manager') @OEHasPackage(['apt']) def test_apt_install_from_repo(self): - self.setup_source_config_for_package_install() - self.pkg('update') - self.pkg('remove --yes run-postinsts-dev') - self.pkg('install --yes --allow-unauthenticated run-postinsts-dev') - self.cleanup_source_config_for_package_install() + if not self.tc.td.get('PACKAGE_FEED_GPG_NAME'): + self.setup_source_config_for_package_install() + self.pkg('update') + self.pkg('remove --yes run-postinsts-dev') + self.pkg('install --yes --allow-unauthenticated run-postinsts-dev') + self.cleanup_source_config_for_package_install() + else: + # when we are here a key has been set to sign the package feed and + # public key and gnupg installed on the image by test_testimage_apt + self.setup_source_config_for_package_install_signed() + self.setup_key() + self.pkg('update') + self.pkg('install --yes run-postinsts-dev') + self.pkg('remove --yes run-postinsts-dev') + self.cleanup_source_config_for_package_install_signed()meta/lib/oeqa/selftest/cases/runtime_test.py +38
-0@@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase): bitbake('core-image-full-cmdline socat') bitbake('-c testimage core-image-full-cmdline') + def test_testimage_apt(self): + """ + Summary: Check package feeds functionality for apt + Expected: 1. Check that remote package feeds can be accessed + Product: oe-core + Author: Ferry Toth <[email protected]> + """ + if get_bb_var('DISTRO') == 'poky-tiny': + self.skipTest('core-image-full-cmdline not buildable for poky-tiny') + + features = 'INHERIT += "testimage"\n' + features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n' + # We don't yet know what the server ip and port will be - they will be patched + # in at the start of the on-image test + features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n' + features += 'EXTRA_IMAGE_FEATURES += "package-management"\n' + features += 'PACKAGE_CLASSES = "package_deb"\n' + # We need gnupg on the target to install keys + features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n' + + bitbake('gnupg-native -c addto_recipe_sysroot') + + # Enable package feed signing + self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-") + self.track_for_cleanup(self.gpg_home) + signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing') + runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True) + features += 'INHERIT += "sign_package_feed"\n' + features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n' + features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase') + features += 'GPG_PATH = "%s"\n' % self.gpg_home + features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home + self.write_config(features) + + # Build core-image-sato and testimage + bitbake('core-image-full-cmdline socat') + bitbake('-c testimage core-image-full-cmdline') + def test_testimage_virgl_gtk_sdl(self): """ Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontendsSignature · 1 line
2.32.0
Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Re: [OE-core] [PATCH v3 1/1] apt: add apt selftest to test signed package feeds
Hi
Op 12-04-2022 om 16:16 schreef Alexandre Belloni:
Quoted 9 lines
Hello,
Quoted 6 lines · Ferry Toth
From: Ferry Toth <[email protected]>
Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.
Quoted 7 lines
Quoted 5 lines
Patches "gpg-sign: Add parameters to gpg signature function" and "
package_manager: sign DEB package feeds" enable signed DEB package feeds. This patch adds a runtime test for apt derived from thetest_testimage_dnftest. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.This went through the autobuilders and it seems this still fails:
That is disappointing.
Quoted 5 lines
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/3437/steps/15/logs/stdio
ERROR: package-index-1.0-r0
do_package_index: Could not get gpg version: Command '['/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg', '--agent-program=/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg-agent|--auto-expand-secmem', '--version', '--no-permission-warning']' returned non-zero exit status 2.
ERROR: Logfile of failure stored in:/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/work/core2-64-poky-linux/package-index/1.0-r0/temp/log.do_package_index.53841
NOTE: recipe package-index-1.0-r0: taskdo_package_index: Failed
In fact package_index is failing, which is outside this patch code.
Quoted 4 lines
ERROR: Task (/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/recipes-core/meta/package-index.bb:do_package_index) failed with exit code '1'
This was ubuntu 16.04 so maybe gpg on the distro is too old (1.4.20) but I'm not sure as I think you are using gnupg-native.
I would have expected gnupg-native, but the log line above shows hosttools is being used. But the same would happen for signed rpm and ipk feeds right?
Did we get the correct one tested? I see 55173d in next and then reverted by Richard. But that was v2.
Quoted 128 lines
Quoted 126 lines
Signed-off-by Ferry Toth <[email protected]>--- meta/lib/oeqa/runtime/cases/apt.py | 38 ++++++++++++++++---- meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++ 2 files changed, 69 insertions(+), 7 deletions(-)meta/lib/oeqa/runtime/cases/apt.py +31
-7@@ -21,7 +21,7 @@ class AptRepoTest(AptTest): @classmethod def setUpClass(cls): - service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all') + service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '') cls.repo_server = HTTPService(service_repo, '0.0.0.0', port=cls.tc.target.server_port, logger=cls.tc.logger)@@ -34,20 +34,44 @@ class AptRepoTest(AptTest): def setup_source_config_for_package_install(self): apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port) apt_get_sourceslist_dir = '/etc/apt/' - self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server)) + self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s/all ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server)) + + def setup_source_config_for_package_install_signed(self): + apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port) + apt_get_sourceslist_dir = '/etc/apt/' + self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server)) def cleanup_source_config_for_package_install(self): apt_get_sourceslist_dir = '/etc/apt/' self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir)) + def cleanup_source_config_for_package_install_signed(self): + apt_get_sourceslist_dir = '/etc/apt/' + self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir)) + + def setup_key(self): + # the key is found on the target /etc/pki/packagefeed-gpg/ + # named PACKAGEFEED-GPG-KEY-poky-branch + self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg')) + @skipIfNotFeature('package-management', 'Test requires package-management to be in IMAGE_FEATURES') @skipIfNotDataVar('IMAGE_PKGTYPE', 'deb', 'DEB is not the primary package manager') @OEHasPackage(['apt']) def test_apt_install_from_repo(self): - self.setup_source_config_for_package_install() - self.pkg('update') - self.pkg('remove --yes run-postinsts-dev') - self.pkg('install --yes --allow-unauthenticated run-postinsts-dev') - self.cleanup_source_config_for_package_install() + if not self.tc.td.get('PACKAGE_FEED_GPG_NAME'): + self.setup_source_config_for_package_install() + self.pkg('update') + self.pkg('remove --yes run-postinsts-dev') + self.pkg('install --yes --allow-unauthenticated run-postinsts-dev') + self.cleanup_source_config_for_package_install() + else: + # when we are here a key has been set to sign the package feed and + # public key and gnupg installed on the image by test_testimage_apt + self.setup_source_config_for_package_install_signed() + self.setup_key() + self.pkg('update') + self.pkg('install --yes run-postinsts-dev') + self.pkg('remove --yes run-postinsts-dev') + self.cleanup_source_config_for_package_install_signed()meta/lib/oeqa/selftest/cases/runtime_test.py +38
-0@@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase): bitbake('core-image-full-cmdline socat') bitbake('-c testimage core-image-full-cmdline') + def test_testimage_apt(self): + """ + Summary: Check package feeds functionality for apt + Expected: 1. Check that remote package feeds can be accessed + Product: oe-core + Author: Ferry Toth <[email protected]> + """ + if get_bb_var('DISTRO') == 'poky-tiny': + self.skipTest('core-image-full-cmdline not buildable for poky-tiny') + + features = 'INHERIT += "testimage"\n' + features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n' + # We don't yet know what the server ip and port will be - they will be patched + # in at the start of the on-image test + features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n' + features += 'EXTRA_IMAGE_FEATURES += "package-management"\n' + features += 'PACKAGE_CLASSES = "package_deb"\n' + # We need gnupg on the target to install keys + features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n' + + bitbake('gnupg-native -c addto_recipe_sysroot') + + # Enable package feed signing + self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-") + self.track_for_cleanup(self.gpg_home) + signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing') + runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True) + features += 'INHERIT += "sign_package_feed"\n' + features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n' + features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase') + features += 'GPG_PATH = "%s"\n' % self.gpg_home + features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home + self.write_config(features) + + # Build core-image-sato and testimage + bitbake('core-image-full-cmdline socat') + bitbake('-c testimage core-image-full-cmdline') + def test_testimage_virgl_gtk_sdl(self): """ Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontendsSignature · 1 line
2.32.0
Re: [OE-core] [PATCH v3 1/1] apt: add apt selftest to test signed package feeds
Quoted 32 lines · Ferry Toth
Hi
Op 12-04-2022 om 16:16 schreef Alexandre Belloni:
Quoted 9 lines
Hello,
Quoted 6 lines · Ferry Toth
From: Ferry Toth <[email protected]>
Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.
Quoted 7 lines
Quoted 5 lines
Patches "gpg-sign: Add parameters to gpg signature function" and "
package_manager: sign DEB package feeds" enable signed DEB package feeds. This patch adds a runtime test for apt derived from thetest_testimage_dnftest. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.This went through the autobuilders and it seems this still fails:
That is disappointing.
Quoted 5 lines
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/3437/steps/15/logs/stdio
ERROR: package-index-1.0-r0
do_package_index: Could not get gpg version: Command '['/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg', '--agent-program=/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg-agent|--auto-expand-secmem', '--version', '--no-permission-warning']' returned non-zero exit status 2.
ERROR: Logfile of failure stored in:/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/work/core2-64-poky-linux/package-index/1.0-r0/temp/log.do_package_index.53841
NOTE: recipe package-index-1.0-r0: taskdo_package_index: FailedIn fact
package_indexis failing, which is outside this patch code.Quoted 2 lines
ERROR: Task (/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/recipes-core/meta/package-index.bb:do_package_index) failed with exit code '1'
Quoted 9 lines
Quoted 2 lines
This was ubuntu 16.04 so maybe gpg on the distro is too old (1.4.20) but I'm not sure as I think you are using gnupg-native.
I would have expected gnupg-native, but the log line above shows hosttools is being used. But the same would happen for signed rpm and ipk feeds right?
Did we get the correct one tested? I see 55173d in next and then reverted by Richard. But that was v2.
This was https://git.yoctoproject.org/poky-contrib/commit/?id=5abda438ce762fc7b8e065e3e9063820c758918e
Just to be sure, I've started on ubuntu1604 both master and this branch, we'll see if this reproduces.
Quoted 124 lines
Quoted 124 lines
Quoted 122 lines
Signed-off-by Ferry Toth <[email protected]>--- meta/lib/oeqa/runtime/cases/apt.py | 38 ++++++++++++++++---- meta/lib/oeqa/selftest/cases/runtime_test.py | 38 ++++++++++++++++++++ 2 files changed, 69 insertions(+), 7 deletions(-)meta/lib/oeqa/runtime/cases/apt.py +31
-7@@ -21,7 +21,7 @@ class AptRepoTest(AptTest): @classmethod def setUpClass(cls): - service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], 'all') + service_repo = os.path.join(cls.tc.td['DEPLOY_DIR_DEB'], '') cls.repo_server = HTTPService(service_repo, '0.0.0.0', port=cls.tc.target.server_port, logger=cls.tc.logger)@@ -34,20 +34,44 @@ class AptRepoTest(AptTest): def setup_source_config_for_package_install(self): apt_get_source_server = 'http://%s:%s/' % (self.tc.target.server_ip, self.repo_server.port) apt_get_sourceslist_dir = '/etc/apt/' - self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server)) + self.target.run('cd %s; echo deb [ allow-insecure=yes ] %s/all ./ > sources.list' % (apt_get_sourceslist_dir, apt_get_source_server)) + + def setup_source_config_for_package_install_signed(self): + apt_get_source_server = 'http:\/\/%s:%s' % (self.tc.target.server_ip, self.repo_server.port) + apt_get_sourceslist_dir = '/etc/apt/' + self.target.run("cd %s; cp sources.list sources.list.bak; sed -i 's/\[trusted=yes\] http:\/\/bogus_ip:bogus_port/%s/g' sources.list" % (apt_get_sourceslist_dir, apt_get_source_server)) def cleanup_source_config_for_package_install(self): apt_get_sourceslist_dir = '/etc/apt/' self.target.run('cd %s; rm sources.list' % (apt_get_sourceslist_dir)) + def cleanup_source_config_for_package_install_signed(self): + apt_get_sourceslist_dir = '/etc/apt/' + self.target.run('cd %s; mv sources.list.bak sources.list' % (apt_get_sourceslist_dir)) + + def setup_key(self): + # the key is found on the target /etc/pki/packagefeed-gpg/ + # named PACKAGEFEED-GPG-KEY-poky-branch + self.target.run('cd %s; apt-key add P*' % ('/etc/pki/packagefeed-gpg')) + @skipIfNotFeature('package-management', 'Test requires package-management to be in IMAGE_FEATURES') @skipIfNotDataVar('IMAGE_PKGTYPE', 'deb', 'DEB is not the primary package manager') @OEHasPackage(['apt']) def test_apt_install_from_repo(self): - self.setup_source_config_for_package_install() - self.pkg('update') - self.pkg('remove --yes run-postinsts-dev') - self.pkg('install --yes --allow-unauthenticated run-postinsts-dev') - self.cleanup_source_config_for_package_install() + if not self.tc.td.get('PACKAGE_FEED_GPG_NAME'): + self.setup_source_config_for_package_install() + self.pkg('update') + self.pkg('remove --yes run-postinsts-dev') + self.pkg('install --yes --allow-unauthenticated run-postinsts-dev') + self.cleanup_source_config_for_package_install() + else: + # when we are here a key has been set to sign the package feed and + # public key and gnupg installed on the image by test_testimage_apt + self.setup_source_config_for_package_install_signed() + self.setup_key() + self.pkg('update') + self.pkg('install --yes run-postinsts-dev') + self.pkg('remove --yes run-postinsts-dev') + self.cleanup_source_config_for_package_install_signed()meta/lib/oeqa/selftest/cases/runtime_test.py +38
-0@@ -162,6 +162,44 @@ class TestImage(OESelftestTestCase): bitbake('core-image-full-cmdline socat') bitbake('-c testimage core-image-full-cmdline') + def test_testimage_apt(self): + """ + Summary: Check package feeds functionality for apt + Expected: 1. Check that remote package feeds can be accessed + Product: oe-core + Author: Ferry Toth <[email protected]> + """ + if get_bb_var('DISTRO') == 'poky-tiny': + self.skipTest('core-image-full-cmdline not buildable for poky-tiny') + + features = 'INHERIT += "testimage"\n' + features += 'TEST_SUITES = "ping ssh apt.AptRepoTest.test_apt_install_from_repo"\n' + # We don't yet know what the server ip and port will be - they will be patched + # in at the start of the on-image test + features += 'PACKAGE_FEED_URIS = "http://bogus_ip:bogus_port"\n' + features += 'EXTRA_IMAGE_FEATURES += "package-management"\n' + features += 'PACKAGE_CLASSES = "package_deb"\n' + # We need gnupg on the target to install keys + features += 'IMAGE_INSTALL:append:pn-core-image-full-cmdline = " gnupg"\n' + + bitbake('gnupg-native -c addto_recipe_sysroot') + + # Enable package feed signing + self.gpg_home = tempfile.mkdtemp(prefix="oeqa-feed-sign-") + self.track_for_cleanup(self.gpg_home) + signing_key_dir = os.path.join(self.testlayer_path, 'files', 'signing') + runCmd('gpgconf --list-dirs --homedir %s; gpg -v --batch --homedir %s --import %s' % (self.gpg_home, self.gpg_home, os.path.join(signing_key_dir, 'key.secret')), native_sysroot=get_bb_var("RECIPE_SYSROOT_NATIVE", "gnupg-native"), shell=True) + features += 'INHERIT += "sign_package_feed"\n' + features += 'PACKAGE_FEED_GPG_NAME = "testuser"\n' + features += 'PACKAGE_FEED_GPG_PASSPHRASE_FILE = "%s"\n' % os.path.join(signing_key_dir, 'key.passphrase') + features += 'GPG_PATH = "%s"\n' % self.gpg_home + features += 'PSEUDO_IGNORE_PATHS .= ",%s"\n' % self.gpg_home + self.write_config(features) + + # Build core-image-sato and testimage + bitbake('core-image-full-cmdline socat') + bitbake('-c testimage core-image-full-cmdline') + def test_testimage_virgl_gtk_sdl(self): """ Summary: Check host-assisted accelerate OpenGL functionality in qemu with gtk and SDL frontendsSignature · 1 line
2.32.0
Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Re: [OE-core] [PATCH v3 1/1] apt: add apt selftest to test signed package feeds
Quoted 39 lines · Alexandre Belloni
Quoted 38 lines · Ferry Toth
Hi
Op 12-04-2022 om 16:16 schreef Alexandre Belloni:
Quoted 9 lines
Hello,
Quoted 6 lines · Ferry Toth
From: Ferry Toth <[email protected]>
Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.
Quoted 7 lines
Quoted 5 lines
Patches "gpg-sign: Add parameters to gpg signature function" and "
package_manager: sign DEB package feeds" enable signed DEB package feeds. This patch adds a runtime test for apt derived from thetest_testimage_dnftest. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.This went through the autobuilders and it seems this still fails:
That is disappointing.
Quoted 5 lines
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/3437/steps/15/logs/stdio
ERROR: package-index-1.0-r0
do_package_index: Could not get gpg version: Command '['/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg', '--agent-program=/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg-agent|--auto-expand-secmem', '--version', '--no-permission-warning']' returned non-zero exit status 2.
ERROR: Logfile of failure stored in:/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/work/core2-64-poky-linux/package-index/1.0-r0/temp/log.do_package_index.53841
NOTE: recipe package-index-1.0-r0: taskdo_package_index: FailedIn fact
package_indexis failing, which is outside this patch code.Quoted 4 lines
ERROR: Task (/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/recipes-core/meta/package-index.bb:do_package_index) failed with exit code '1'
This was ubuntu 16.04 so maybe gpg on the distro is too old (1.4.20) but I'm not sure as I think you are using gnupg-native.
I would have expected gnupg-native, but the log line above shows hosttools is being used. But the same would happen for signed rpm and ipk feeds right?
Quoted 8 lines
Quoted 3 lines
Did we get the correct one tested? I see 55173d in next and then reverted by Richard. But that was v2.
This was https://git.yoctoproject.org/poky-contrib/commit/?id=5abda438ce762fc7b8e065e3e9063820c758918e
Just to be sure, I've started on ubuntu1604 both master and this branch, we'll see if this reproduces.
Firstly, this is occurring in the newly added test so this is being triggered by the new code. I suspect what is happening is that gnupg-native isn't being built before the test and this means that it is falling back to the system gpg. The system gpg is too old on that worker so it fails.
You can probably reproduce locally by not having a gpg on your build system (move it out the way temporarily?).
If I'm right (and I'm just guessing), the fix is to add the missing dependency to ensure gpg is one we've built.
Cheers,
Richard
Re: [OE-core] [PATCH v3 1/1] apt: add apt selftest to test signed package feeds
Hi,
Op 12-04-2022 om 23:51 schreef Richard Purdie:
Quoted 37 lines · Alexandre Belloni
Quoted 36 lines · Ferry Toth
Quoted 35 lines
Hi
Op 12-04-2022 om 16:16 schreef Alexandre Belloni:
Quoted 9 lines
Hello,
Quoted 6 lines · Ferry Toth
From: Ferry Toth <[email protected]>
Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.
Quoted 7 lines
Quoted 5 lines
Patches "gpg-sign: Add parameters to gpg signature function" and "
package_manager: sign DEB package feeds" enable signed DEB package feeds. This patch adds a runtime test for apt derived from thetest_testimage_dnftest. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.This went through the autobuilders and it seems this still fails:
That is disappointing.
Quoted 5 lines
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/3437/steps/15/logs/stdio
ERROR: package-index-1.0-r0
do_package_index: Could not get gpg version: Command '['/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg', '--agent-program=/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg-agent|--auto-expand-secmem', '--version', '--no-permission-warning']' returned non-zero exit status 2.
ERROR: Logfile of failure stored in:/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/work/core2-64-poky-linux/package-index/1.0-r0/temp/log.do_package_index.53841
NOTE: recipe package-index-1.0-r0: taskdo_package_index: FailedIn fact
package_indexis failing, which is outside this patch code.Quoted 4 lines
ERROR: Task (/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/recipes-core/meta/package-index.bb:do_package_index) failed with exit code '1'
This was ubuntu 16.04 so maybe gpg on the distro is too old (1.4.20) but I'm not sure as I think you are using gnupg-native.
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
I would have expected gnupg-native, but the log line above shows hosttools is being used. But the same would happen for signed rpm and ipk feeds right?
Did we get the correct one tested? I see 55173d in next and then reverted by Richard. But that was v2.
This was https://git.yoctoproject.org/poky-contrib/commit/?id=5abda438ce762fc7b8e065e3e9063820c758918e
This is the correct one.
Quoted 7 lines
Quoted 2 lines
Just to be sure, I've started on ubuntu1604 both master and this branch, we'll see if this reproduces.
Firstly, this is occurring in the newly added test so this is being triggered by the new code. I suspect what is happening is that gnupg-native isn't being built before the test and this means that it is falling back to the system gpg. The system gpg is too old on that worker so it fails.
Certainly
Quoted 2 lines
You can probably reproduce locally by not having a gpg on your build system (move it out the way temporarily?).
Thanks for the tip. Not sure if I can remove the package, but IIUC it's the executable that needs to be present so I can just move it out of the way.
Quoted 2 lines
If I'm right (and I'm just guessing), the fix is to add the missing dependency to ensure gpg is one we've built.
I know how to add dependency in a recipe, but where to add here?
I already have 'bitbake('gnupg-native -c addto_recipe_sysroot')' Should I run 'bitbake('gnupg-native')' before that?
I copied these lines from test_testimage_dnf, shouldn't that have similar problems?
Quoted 6 lines
Cheers,
Richard
Re: [OE-core] [PATCH v3 1/1] apt: add apt selftest to test signed package feeds
Quoted 70 lines · Ferry Toth
Hi,
Op 12-04-2022 om 23:51 schreef Richard Purdie:
Quoted 37 lines · Alexandre Belloni
Quoted 36 lines · Ferry Toth
Quoted 35 lines
Hi
Op 12-04-2022 om 16:16 schreef Alexandre Belloni:
Quoted 9 lines
Hello,
Quoted 6 lines · Ferry Toth
From: Ferry Toth <[email protected]>
Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.
Quoted 7 lines
Quoted 5 lines
Patches "gpg-sign: Add parameters to gpg signature function" and "
package_manager: sign DEB package feeds" enable signed DEB package feeds. This patch adds a runtime test for apt derived from thetest_testimage_dnftest. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.This went through the autobuilders and it seems this still fails:
That is disappointing.
Quoted 5 lines
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/3437/steps/15/logs/stdio
ERROR: package-index-1.0-r0
do_package_index: Could not get gpg version: Command '['/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg', '--agent-program=/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg-agent|--auto-expand-secmem', '--version', '--no-permission-warning']' returned non-zero exit status 2.
ERROR: Logfile of failure stored in:/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/work/core2-64-poky-linux/package-index/1.0-r0/temp/log.do_package_index.53841
NOTE: recipe package-index-1.0-r0: taskdo_package_index: FailedIn fact
package_indexis failing, which is outside this patch code.Quoted 4 lines
ERROR: Task (/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/recipes-core/meta/package-index.bb:do_package_index) failed with exit code '1'
This was ubuntu 16.04 so maybe gpg on the distro is too old (1.4.20) but I'm not sure as I think you are using gnupg-native.
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
I would have expected gnupg-native, but the log line above shows hosttools is being used. But the same would happen for signed rpm and ipk feeds right?
Did we get the correct one tested? I see 55173d in next and then reverted by Richard. But that was v2.
This was https://git.yoctoproject.org/poky-contrib/commit/?id=5abda438ce762fc7b8e065e3e9063820c758918e
This is the correct one.
Quoted 7 lines
Quoted 2 lines
Just to be sure, I've started on ubuntu1604 both master and this branch, we'll see if this reproduces.
Firstly, this is occurring in the newly added test so this is being triggered by the new code. I suspect what is happening is that gnupg-native isn't being built before the test and this means that it is falling back to the system gpg. The system gpg is too old on that worker so it fails.
Certainly
Quoted 2 lines
You can probably reproduce locally by not having a gpg on your build system (move it out the way temporarily?).
Thanks for the tip. Not sure if I can remove the package, but IIUC it's the executable that needs to be present so I can just move it out of the way.
Quoted 2 lines
If I'm right (and I'm just guessing), the fix is to add the missing dependency to ensure gpg is one we've built.
Quoted 8 lines
I know how to add dependency in a recipe, but where to add here?
I already have '
bitbake('gnupg-native -c addto_recipe_sysroot')' Should I run 'bitbake('gnupg-native')' before that?I copied these lines from
test_testimage_dnf, shouldn't that have similar problems?
sign_rpm.bbclass has PACKAGE_WRITE_DEPS += "gnupg-native", doesn't that solve this issue?
Signature · 3 lines
Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Re: [OE-core] [PATCH v3 1/1] apt: add apt selftest to test signed package feeds
Quoted 76 lines · Alexandre Belloni
Quoted 75 lines · Ferry Toth
Hi,
Op 12-04-2022 om 23:51 schreef Richard Purdie:
Quoted 37 lines · Alexandre Belloni
Quoted 36 lines · Ferry Toth
Quoted 35 lines
Hi
Op 12-04-2022 om 16:16 schreef Alexandre Belloni:
Quoted 9 lines
Hello,
Quoted 6 lines · Ferry Toth
From: Ferry Toth <[email protected]>
Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.
Quoted 7 lines
Quoted 5 lines
Patches "gpg-sign: Add parameters to gpg signature function" and "
package_manager: sign DEB package feeds" enable signed DEB package feeds. This patch adds a runtime test for apt derived from thetest_testimage_dnftest. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.This went through the autobuilders and it seems this still fails:
That is disappointing.
Quoted 5 lines
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/3437/steps/15/logs/stdio
ERROR: package-index-1.0-r0
do_package_index: Could not get gpg version: Command '['/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg', '--agent-program=/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg-agent|--auto-expand-secmem', '--version', '--no-permission-warning']' returned non-zero exit status 2.
ERROR: Logfile of failure stored in:/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/work/core2-64-poky-linux/package-index/1.0-r0/temp/log.do_package_index.53841
NOTE: recipe package-index-1.0-r0: taskdo_package_index: FailedIn fact
package_indexis failing, which is outside this patch code.Quoted 4 lines
ERROR: Task (/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/recipes-core/meta/package-index.bb:do_package_index) failed with exit code '1'
This was ubuntu 16.04 so maybe gpg on the distro is too old (1.4.20) but I'm not sure as I think you are using gnupg-native.
Quoted 8 lines
Quoted 8 lines
Quoted 6 lines
I would have expected gnupg-native, but the log line above shows hosttools is being used. But the same would happen for signed rpm and ipk feeds right?
Did we get the correct one tested? I see 55173d in next and then reverted by Richard. But that was v2.
This was https://git.yoctoproject.org/poky-contrib/commit/?id=5abda438ce762fc7b8e065e3e9063820c758918e
This is the correct one.
Quoted 7 lines
Quoted 2 lines
Just to be sure, I've started on ubuntu1604 both master and this branch, we'll see if this reproduces.
Firstly, this is occurring in the newly added test so this is being triggered by the new code. I suspect what is happening is that gnupg-native isn't being built before the test and this means that it is falling back to the system gpg. The system gpg is too old on that worker so it fails.
Certainly
Quoted 2 lines
You can probably reproduce locally by not having a gpg on your build system (move it out the way temporarily?).
Thanks for the tip. Not sure if I can remove the package, but IIUC it's the executable that needs to be present so I can just move it out of the way.
Quoted 2 lines
If I'm right (and I'm just guessing), the fix is to add the missing dependency to ensure gpg is one we've built.
I know how to add dependency in a recipe, but where to add here?
I already have '
bitbake('gnupg-native -c addto_recipe_sysroot')' Should I run 'bitbake('gnupg-native')' before that?
Quoted 6 lines
Quoted 3 lines
I copied these lines from
test_testimage_dnf, shouldn't that have similar problems?sign_rpm.bbclass has
PACKAGE_WRITE_DEPS+= "gnupg-native", doesn't that solve this issue?
Perhaps sign_package_feed.bbclass needs something like:
PACKAGEINDEXDEPS += "gnupg-native:do_populate_sysroot"
I'm not sure why/how it works in the rpm case but it does seem like the dependency is missing in the deb one.
Cheers,
Richard
Re: [OE-core] [PATCH v3 1/1] apt: add apt selftest to test signed package feeds
Hi,
Op 13-04-2022 om 09:00 schreef Richard Purdie:
Quoted 55 lines · Alexandre Belloni
Quoted 54 lines · Ferry Toth
Quoted 53 lines
Hi,
Op 12-04-2022 om 23:51 schreef Richard Purdie:
Quoted 33 lines · Alexandre Belloni
Quoted 32 lines · Ferry Toth
Quoted 31 lines
Hi
Op 12-04-2022 om 16:16 schreef Alexandre Belloni:
Quoted 9 lines
Hello,
Quoted 6 lines · Ferry Toth
From: Ferry Toth <[email protected]>
Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.
Quoted 6 lines
Quoted 5 lines
Patches "gpg-sign: Add parameters to gpg signature function" and "
package_manager: sign DEB package feeds" enable signed DEB package feeds. This patch adds a runtime test for apt derived from thetest_testimage_dnftest. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.This went through the autobuilders and it seems this still fails:
That is disappointing.
Quoted 5 lines
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/3437/steps/15/logs/stdio
ERROR: package-index-1.0-r0
do_package_index: Could not get gpg version: Command '['/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg', '--agent-program=/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg-agent|--auto-expand-secmem', '--version', '--no-permission-warning']' returned non-zero exit status 2.
ERROR: Logfile of failure stored in:/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/work/core2-64-poky-linux/package-index/1.0-r0/temp/log.do_package_index.53841
NOTE: recipe package-index-1.0-r0: taskdo_package_index: FailedIn fact
package_indexis failing, which is outside this patch code.Quoted 4 lines
ERROR: Task (/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/recipes-core/meta/package-index.bb:do_package_index) failed with exit code '1'
This was ubuntu 16.04 so maybe gpg on the distro is too old (1.4.20) but I'm not sure as I think you are using gnupg-native.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
I would have expected gnupg-native, but the log line above shows hosttools is being used. But the same would happen for signed rpm and ipk feeds right?
Did we get the correct one tested? I see 55173d in next and then reverted by Richard. But that was v2.
This was https://git.yoctoproject.org/poky-contrib/commit/?id=5abda438ce762fc7b8e065e3e9063820c758918e
This is the correct one.
Quoted 6 lines
Quoted 2 lines
Just to be sure, I've started on ubuntu1604 both master and this branch, we'll see if this reproduces.
Firstly, this is occurring in the newly added test so this is being triggered by the new code. I suspect what is happening is that gnupg-native isn't being built before the test and this means that it is falling back to the system gpg. The system gpg is too old on that worker so it fails.
Certainly
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
You can probably reproduce locally by not having a gpg on your build system (move it out the way temporarily?).
Thanks for the tip. Not sure if I can remove the package, but IIUC it's the executable that needs to be present so I can just move it out of the way.
Quoted 2 lines
If I'm right (and I'm just guessing), the fix is to add the missing dependency to ensure gpg is one we've built.
You are right
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
I know how to add dependency in a recipe, but where to add here?
I already have '
bitbake('gnupg-native -c addto_recipe_sysroot')' Should I run 'bitbake('gnupg-native')' before that?
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
I copied these lines from
test_testimage_dnf, shouldn't that have similar problems?sign_rpm.bbclass has
PACKAGE_WRITE_DEPS+= "gnupg-native", doesn't that solve this issue?Perhaps sign_package_feed.bbclass needs something like:
PACKAGEINDEXDEPS += "gnupg-native:do_populate_sysroot"
I added this to the end of 'meta/classes/package_deb.bbclass' and that works.
Do you agree this is the right place?
Should I squash with this patch or send in as a separate patch (fixes 0b4231b5 <https://git.yoctoproject.org/poky/commit/?id=0b4231b597618e18668b8340f4209cd364b2b2d0> "package_manager: sign DEB package feeds")?
Quoted 8 lines
I'm not sure why/how it works in the rpm case but it does seem like the dependency is missing in the deb one.
Cheers,
Richard
Re: [OE-core] [PATCH v3 1/1] apt: add apt selftest to test signed package feeds
Quoted 76 lines · Ferry Toth
Hi,
Op 13-04-2022 om 09:00 schreef Richard Purdie:
Quoted 55 lines · Alexandre Belloni
Quoted 54 lines · Ferry Toth
Quoted 53 lines
Hi,
Op 12-04-2022 om 23:51 schreef Richard Purdie:
Quoted 33 lines · Alexandre Belloni
Quoted 32 lines · Ferry Toth
Quoted 31 lines
Hi
Op 12-04-2022 om 16:16 schreef Alexandre Belloni:
Quoted 9 lines
Hello,
Quoted 6 lines · Ferry Toth
From: Ferry Toth <[email protected]>
Since Gatesgarth apt (1.8.2) has become more strict and doesn’t allow unsigned repositories by default. Currently when building images this requirement is worked around by using [allow-insecure=yes] and equivalently when performing selftest.
Quoted 6 lines
Quoted 5 lines
Patches "gpg-sign: Add parameters to gpg signature function" and "
package_manager: sign DEB package feeds" enable signed DEB package feeds. This patch adds a runtime test for apt derived from thetest_testimage_dnftest. It creates a signed deb package feed, runs a qemu image to install the key and performs some package management. To be able to install the key the gnupg package is added to the testimage.This went through the autobuilders and it seems this still fails:
That is disappointing.
Quoted 5 lines
https://autobuilder.yoctoproject.org/typhoon/#/builders/87/builds/3437/steps/15/logs/stdio
ERROR: package-index-1.0-r0
do_package_index: Could not get gpg version: Command '['/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg', '--agent-program=/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/hosttools/gpg-agent|--auto-expand-secmem', '--version', '--no-permission-warning']' returned non-zero exit status 2.
ERROR: Logfile of failure stored in:/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/build-st-34525/tmp/work/core2-64-poky-linux/package-index/1.0-r0/temp/log.do_package_index.53841
NOTE: recipe package-index-1.0-r0: taskdo_package_index: FailedIn fact
package_indexis failing, which is outside this patch code.Quoted 4 lines
ERROR: Task (/home/pokybuild/yocto-worker/oe-selftest-ubuntu/build/meta/recipes-core/meta/package-index.bb:do_package_index) failed with exit code '1'
This was ubuntu 16.04 so maybe gpg on the distro is too old (1.4.20) but I'm not sure as I think you are using gnupg-native.
Quoted 7 lines
Quoted 7 lines
Quoted 6 lines
I would have expected gnupg-native, but the log line above shows hosttools is being used. But the same would happen for signed rpm and ipk feeds right?
Did we get the correct one tested? I see 55173d in next and then reverted by Richard. But that was v2.
This was https://git.yoctoproject.org/poky-contrib/commit/?id=5abda438ce762fc7b8e065e3e9063820c758918e
This is the correct one.
Quoted 6 lines
Quoted 2 lines
Just to be sure, I've started on ubuntu1604 both master and this branch, we'll see if this reproduces.
Firstly, this is occurring in the newly added test so this is being triggered by the new code. I suspect what is happening is that gnupg-native isn't being built before the test and this means that it is falling back to the system gpg. The system gpg is too old on that worker so it fails.
Certainly
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
You can probably reproduce locally by not having a gpg on your build system (move it out the way temporarily?).
Thanks for the tip. Not sure if I can remove the package, but IIUC it's the executable that needs to be present so I can just move it out of the way.
Quoted 2 lines
If I'm right (and I'm just guessing), the fix is to add the missing dependency to ensure gpg is one we've built.
You are right
Quoted 10 lines
Quoted 10 lines
Quoted 8 lines
I know how to add dependency in a recipe, but where to add here?
I already have '
bitbake('gnupg-native -c addto_recipe_sysroot')' Should I run 'bitbake('gnupg-native')' before that?I copied these lines from
test_testimage_dnf, shouldn't that have similar problems?sign_rpm.bbclass has
PACKAGE_WRITE_DEPS+= "gnupg-native", doesn't that solve this issue?
Quoted 8 lines
Quoted 3 lines
Perhaps sign_package_feed.bbclass needs something like:
PACKAGEINDEXDEPS += "gnupg-native:do_populate_sysroot"
I added this to the end of 'meta/classes/package_deb.bbclass' and that works.
Do you agree this is the right place?
No. That builds gpg pieces even when signing isn't enabled so I don't think that is right.
Quoted 4 lines
Should I squash with this patch or send in as a separate patch (fixes
0b4231b5
<https://git.yoctoproject.org/poky/commit/?id=0b4231b597618e18668b8340f4209cd364b2b2d0>
"package_manager: sign DEB package feeds")?
I think we can likely make this change a separate commit since it will have it's own explanation with it.
Cheers,
Richard