Skip to contentSkip to conversation

gpg-sign: Add parameters to gpg signature function

PATCH Superseded Ferry Toth 3 participants 13 messages Mar 22, 2022 – Mar 28, 2022

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Ferry Toth

author
#0 lore raw

PATCH 1/2 gpg-sign: Add parameters to gpg signature function

From: Xavier Berger <[email protected]>

output_suffix: If defined, add output_suffix as file name extension.
use_sha256: If True, use sha256 for gpg as digest algorithm

---
 meta/lib/oe/gpg_sign.py | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
meta/lib/oe/gpg_sign.py +5 -1
index 1bce6cb792..aa9bb49f2c 100644
@@ -58,7 +58,7 @@ class LocalSigner(object):
         for i in range(0, len(files), sign_chunk):
             subprocess.check_output(shlex.split(cmd + ' '.join(files[i:i+sign_chunk])), stderr=subprocess.STDOUT)
 
-    def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True):
+    def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True, output_suffix=None, use_sha256=False):
         """Create a detached signature of a file"""
 
         if passphrase_file and passphrase:
@@ -71,6 +71,10 @@ class LocalSigner(object):
             cmd += ['--homedir', self.gpg_path]
         if armor:
             cmd += ['--armor']
+        if output_suffix:
+            cmd += ['-o', input_file + "." + output_suffix]
+        if use_sha256:
+            cmd += ['--digest-algo', "SHA256"]
 
         #gpg > 2.1 supports password pipes only through the loopback interface
         #gpg < 2.1 errors out if given unknown parameters
Signature · 1 line
2.32.0
author
#1 lore raw

PATCH 2/2 package_manager: sign DEB package feeds

From: Ferry Toth <[email protected]>

Implement debian package repository signature.
For each Release file created in repository subdirectory, a signature Release.gpg is created.

Signature is performed using gpg backend when the following variables are set in local.conf:
PACKAGE_CLASSES += "sign_package_feed" PACKAGE_FEED_GPG_NAME = "<Id of GPG key>" PACKAGE_FEED_GPG_PASSPHRASE_FILE="<path to password file>"

---
 meta/lib/oe/package_manager/deb/__init__.py | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)
meta/lib/oe/package_manager/deb/__init__.py +16 -3
index 9f112ae25b..86ddb130ad 100644
@@ -53,6 +53,7 @@ class DpkgIndexer(Indexer):
 
         index_cmds = []
         deb_dirs_found = False
+        index_sign_files = set()
         for arch in arch_list:
             arch_dir = os.path.join(self.deploy_dir, arch)
             if not os.path.isdir(arch_dir):
@@ -62,7 +63,10 @@ class DpkgIndexer(Indexer):
 
             cmd += "%s -fcn Packages > Packages.gz;" % gzip
 
-            with open(os.path.join(arch_dir, "Release"), "w+") as release:
+            release_file = os.path.join(arch_dir, "Release")
+            index_sign_files.add(release_file)
+
+            with open(release_file, "w+") as release:
                 release.write("Label: %s\n" % arch)
 
             cmd += "PSEUDO_UNLOAD=1 %s release . >> Release" % apt_ftparchive
@@ -76,8 +80,17 @@ class DpkgIndexer(Indexer):
             return
 
         oe.utils.multiprocess_launch(create_index, index_cmds, self.d)
-        if self.d.getVar('PACKAGE_FEED_SIGN') == '1':
-            raise NotImplementedError('Package feed signing not implementd for dpkg')
+        if self.d.getVar('PACKAGE_FEED_SIGN', True) == '1':
+            signer = get_signer(self.d, self.d.getVar('PACKAGE_FEED_GPG_BACKEND', True))
+        else:
+            signer = None
+        if signer:
+            for f in index_sign_files:
+                signer.detach_sign(f,
+                                   self.d.getVar('PACKAGE_FEED_GPG_NAME', True),
+                                   self.d.getVar('PACKAGE_FEED_GPG_PASSPHRASE_FILE', True),
+                                   output_suffix="gpg",
+                                   use_sha256=True)
 
 class PMPkgsList(PkgsList):
 
Signature · 1 line
2.32.0
author
#2 lore raw
In reply to Ferry Toth

Hi Richard,

I forgot to add a cover letter, sorry for that. The 2 patches  together implement DEB repository signing.

This is necessary since Gatesgarth |apt| (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.

It is possible to override this behavior |but||| is more work then to enable signed DEB repositories. These patches makes DEB a first class citizen as IPK and RPM.

Patches have been in use in meta-intel-edison since Gatesgarth, see https://edison-fw.github.io/meta-intel-edison/5.0-Creating-a-deb-repository.html\

Ferry

Op 22-03-2022 om 22:19 schreef Ferry Toth:

Quoted 35 lines

From: Xavier Berger <[email protected]>

output_suffix: If defined, add output_suffix as file name extension.
use_sha256: If True, use sha256 for gpg as digest algorithm

---
  meta/lib/oe/gpg_sign.py | 6 +++++-
  1 file changed, 5 insertions(+), 1 deletion(-)
meta/lib/oe/gpg_sign.py +5 -1
index 1bce6cb792..aa9bb49f2c 100644
@@ -58,7 +58,7 @@ class LocalSigner(object):
          for i in range(0, len(files), sign_chunk):
              subprocess.check_output(shlex.split(cmd + ' '.join(files[i:i+sign_chunk])), stderr=subprocess.STDOUT)
 
-    def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True):
+    def detach_sign(self, input_file, keyid, passphrase_file, passphrase=None, armor=True, output_suffix=None, use_sha256=False):
          """Create a detached signature of a file"""
 
          if passphrase_file and passphrase:
@@ -71,6 +71,10 @@ class LocalSigner(object):
              cmd += ['--homedir', self.gpg_path]
          if armor:
              cmd += ['--armor']
+        if output_suffix:
+            cmd += ['-o', input_file + "." + output_suffix]
+        if use_sha256:
+            cmd += ['--digest-algo', "SHA256"]
 
          #gpg > 2.1 supports password pipes only through the loopback interface
          #gpg < 2.1 errors out if given unknown parameters
#3 lore raw
Quoted 8 lines · Ferry Toth

Hi Richard,

I forgot to add a cover letter, sorry for that. The 2 patches  together implement DEB repository signing.

This is necessary since Gatesgarth |apt| (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.

Quoted 6 lines

It is possible to override this behavior |but||| is more work then to enable signed DEB repositories. These patches makes DEB a first class citizen as IPK and RPM.

Patches have been in use in meta-intel-edison since Gatesgarth, see
https://edison-fw.github.io/meta-intel-edison/5.0-Creating-a-deb-repository.html\

What puzzles me is that we can build root filesystems using apt, we test this on the autobuilder. Saying repositories are broken since gatesgarth therefore seems confusing in the commit message.

I guess we must configure apt to override that during the rootfs process and likely an end user with a remote feed could do the same, possibly with a warning from apt?

I'm also worried that there isn't any automated testing of this change. The reason I worry is that since we don't show any testing failures right now, there is clearly a hole in our automated testing coverage and there is no guarantee that this feature will keep working. It is these smaller corner case issues which tend to make or break the project's experience as if a feature is present, people expect it to work. Can we improve the testing situation?

Cheers,

Richard

author
#4 lore raw

Hi,

Op 24-03-2022 om 09:12 schreef Richard Purdie:

Quoted 13 lines · Ferry Toth
Quoted 12 lines

Hi Richard,

I forgot to add a cover letter, sorry for that. The 2 patches  together implement DEB repository signing.

This is necessary since Gatesgarth |apt| (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.

It is possible to override this behavior |but||| is more work then to enable signed DEB repositories. These patches makes DEB a first class citizen as IPK and RPM.

Quoted 5 lines
Quoted 2 lines

Patches have been in use in meta-intel-edison since Gatesgarth, see
https://edison-fw.github.io/meta-intel-edison/5.0-Creating-a-deb-repository.html\

What puzzles me is that we can build root filesystems using apt, we test this on the autobuilder. Saying repositories are broken since gatesgarth therefore seems confusing in the commit message.

Good question. When I (meta-intel-edison) build the rootfs using DEB's it just works.

Could it be that during rootfs build dpkg is used and not apt? I think I have seen that in the logs.

Of course apt uses dpkg to install a package as well, but it refuses to download the package from a repo when it's not signed.

Quoted 3 lines

I guess we must configure apt to override that during the rootfs process and likely an end user with a remote feed could do the same, possibly with a warning from apt?

I believe there is no issue during rootfs generation.

Quoted 6 lines

I'm also worried that there isn't any automated testing of this change. The reason I worry is that since we don't show any testing failures right now, there is clearly a hole in our automated testing coverage and there is no guarantee that this feature will keep working. It is these smaller corner case issues which tend to make or break the project's experience as if a feature is present, people expect it to work. Can we improve the testing situation?

It doesn't seem to be a particularly volatile area in the code. I refreshed Xavier's patches for Gatesgarth, and am actively using unchanged patch on Honisiter.

I don't know how the automated testing is working but I guess for RPM a repo is generated using a small layer? And then tested on a qemu running the rootfs?

Should be almost same for deb/apt, maybe could be modified from rpm test?

Point is: currently deb is documented as a feasible package format to generate a repo. But it really is not without these signing patches. So we could either deprecate deb's (no, no please don't) or fix it.

These patches fix it. With or without automated testing, it is already better then the current situation.

Quoted 6 lines

Cheers,

Richard

#5 lore raw
Quoted 20 lines · Ferry Toth
Quoted 12 lines · Ferry Toth
Quoted 11 lines

I forgot to add a cover letter, sorry for that. The 2 patches  together implement DEB repository signing.

This is necessary since Gatesgarth |apt| (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.

It is possible to override this behavior |but||| is more work then to enable signed DEB repositories. These patches makes DEB a first class citizen as IPK and RPM.

Quoted 8 lines
Quoted 3 lines

Patches have been in use in meta-intel-edison since Gatesgarth, see
https://edison-fw.github.io/meta-intel-edison/5.0-Creating-a-deb-
repository.html\

What puzzles me is that we can build root filesystems using apt, we test this on
the autobuilder. Saying repositories are broken since gatesgarth therefore seems
confusing in the commit message.

Quoted 4 lines

Good question. When I (meta-intel-edison) build the rootfs using DEB's it just works.
Could it be that during rootfs build dpkg is used and not apt? I think I have seen that in the logs.

It definitely uses apt.

Quoted 2 lines

Of course apt uses dpkg to install a package as well, but it refuses to download the package from a repo when it's not signed.

Perhaps the difference is the packages are local and not remote?

Quoted 15 lines
Quoted 4 lines

I guess we must configure apt to override that during the rootfs process and likely an end user with a remote feed could do the same, possibly with a warning
from apt?

 I believe there is no issue during rootfs generation.

Quoted 9 lines

I'm also worried that there isn't any automated testing of this change. The reason I worry is that since we don't show any testing failures right now, there
is clearly a hole in our automated testing coverage and there is no guarantee
that this feature will keep working. It is these smaller corner case issues which tend to make or break the project's experience as if a feature is present,
people expect it to work. Can we improve the testing situation?

Quoted 7 lines

It doesn't seem to be a particularly volatile area in the code. I refreshed Xavier's patches for Gatesgarth, and am actively using unchanged patch on Honisiter.
I don't know how the automated testing is working but I guess for RPM a repo is generated using a small layer? And then tested on a qemu running the rootfs?
Should be almost same for deb/apt, maybe could be modified from rpm test?

I think the rpm test is test_testimage_dnf in meta/lib/oeqa/selftest/cases/runtime_test.py. You'd run it with:

oe-selftest -r runtime_test.TestImage.test_testimage_dnf

Quoted 5 lines

Point is: currently deb is documented as a feasible package format to generate a repo. But it really is not without these signing patches. So we could either deprecate deb's (no, no please don't) or fix it.
These patches fix it. With or without automated testing, it is already better then the current situation.

I'm not sure it is. The project gains yet another set of config options which we don't have tests for and the maintainer stress levels in trying to keep it all working just get worse.

We have a huge number of ways people can configure the builds. The only way the project manages to keep all of this working is through automated testing, there is simply no other way to do it. I appreciate adding tests is a pain and nobody likes doing it. It does however let the project stay functional for everyone's diverse use cases.

There are all kinds of patches I could take because the improve some corner case. In most cases we don't know what else they might break or whether that code continues to be used or stays working. I therefore do really want tests for new configurations.

Who should write those tests? If you don't/can't as some actively using the feature, it means someone else has to, but who?

I've not decided what to do with the patches to be honest but the merge and not bother with tests argument doesn't sit well with me.

Cheers,

Richard

author
#6 lore raw

Hi

Op 24-03-2022 om 13:03 schreef Richard Purdie:

Quoted 52 lines · Ferry Toth
Quoted 19 lines
Quoted 11 lines · Ferry Toth
Quoted 10 lines

I forgot to add a cover letter, sorry for that. The 2 patches  together implement DEB repository signing.

This is necessary since Gatesgarth |apt| (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.

It is possible to override this behavior |but||| is more work then to enable signed DEB repositories. These patches makes DEB a first class citizen as IPK and RPM.

Quoted 8 lines
Quoted 3 lines

Patches have been in use in meta-intel-edison since Gatesgarth, see
https://edison-fw.github.io/meta-intel-edison/5.0-Creating-a-deb-
repository.html\

What puzzles me is that we can build root filesystems using apt, we test this on
the autobuilder. Saying repositories are broken since gatesgarth therefore seems
confusing in the commit message.

Quoted 4 lines

Good question. When I (meta-intel-edison) build the rootfs using DEB's it just works.
Could it be that during rootfs build dpkg is used and not apt? I think I have seen that in the logs.

It definitely uses apt.

Quoted 2 lines

Of course apt uses dpkg to install a package as well, but it refuses to download the package from a repo when it's not signed.

Perhaps the difference is the packages are local and not remote?

Quoted 15 lines
Quoted 4 lines

I guess we must configure apt to override that during the rootfs process and likely an end user with a remote feed could do the same, possibly with a warning
from apt?

 I believe there is no issue during rootfs generation.

Quoted 9 lines

I'm also worried that there isn't any automated testing of this change. The reason I worry is that since we don't show any testing failures right now, there
is clearly a hole in our automated testing coverage and there is no guarantee
that this feature will keep working. It is these smaller corner case issues which tend to make or break the project's experience as if a feature is present,
people expect it to work. Can we improve the testing situation?

Quoted 7 lines

It doesn't seem to be a particularly volatile area in the code. I refreshed Xavier's patches for Gatesgarth, and am actively using unchanged patch on Honisiter.
I don't know how the automated testing is working but I guess for RPM a repo is generated using a small layer? And then tested on a qemu running the rootfs?
Should be almost same for deb/apt, maybe could be modified from rpm test?

Quoted 4 lines

I think the rpm test is test_testimage_dnf in meta/lib/oeqa/selftest/cases/runtime_test.py. You'd run it with:

oe-selftest -r runtime_test.TestImage.test_testimage_dnf

I'll have a look.

Quoted 15 lines
Quoted 5 lines

Point is: currently deb is documented as a feasible package format to generate a repo. But it really is not without these signing patches. So we could either deprecate deb's (no, no please don't) or fix it.
These patches fix it. With or without automated testing, it is already better then the current situation.

I'm not sure it is. The project gains yet another set of config options which we don't have tests for and the maintainer stress levels in trying to keep it all working just get worse.

We have a huge number of ways people can configure the builds. The only way the project manages to keep all of this working is through automated testing, there is simply no other way to do it. I appreciate adding tests is a pain and nobody likes doing it. It does however let the project stay functional for everyone's diverse use cases.

Quoted 4 lines

There are all kinds of patches I could take because the improve some corner case. In most cases we don't know what else they might break or whether that code continues to be used or stays working. I therefore do really want tests for new configurations.

It's not an improvement but a fix for an existing config. Even if it would break in the future it can't be worse then it is now.

Quoted 2 lines

Who should write those tests? If you don't/can't as some actively using the feature, it means someone else has to, but who?

I do understand.

Quoted 7 lines

I've not decided what to do with the patches to be honest but the merge and not bother with tests argument doesn't sit well with me.

Cheers,

Richard

author
#7 lore raw

Hi

Op 24-03-2022 om 16:36 schreef Ferry Toth:

Quoted 24 lines

Hi

Op 24-03-2022 om 13:03 schreef Richard Purdie:

Quoted 20 lines · Ferry Toth
Quoted 19 lines
Quoted 19 lines · Ferry Toth
Quoted 6 lines

I forgot to add a cover letter, sorry for that. The 2 patches  together implement DEB repository signing.

This is necessary since Gatesgarth |apt| (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.

Quoted 7 lines

It is possible to override this behavior |but||| is more work then to enable signed DEB repositories. These patches makes DEB a first class citizen as IPK and RPM.

Patches have been in use in meta-intel-edison since Gatesgarth, see
https://edison-fw.github.io/meta-intel-edison/5.0-Creating-a-deb-
repository.html\

What puzzles me is that we can build root filesystems using apt, we test this on
the autobuilder. Saying repositories are broken since gatesgarth therefore seems
confusing in the commit message.

The Development Task manual in 3.22.4.3.3 names it a repository, as I believe is common for a distributions server holding pre-built packages. But I could change the name to package feed if that would be better.

What I meant to say is that apt requires signed package feeds since 1.8.2 and disabling that is a workaround.

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

Good question. When I (meta-intel-edison) build the rootfs using DEB's it just works.
Could it be that during rootfs build dpkg is used and not apt? I think I have seen that in the logs.

It definitely uses apt.

Checking logs I see you are correct.

Quoted 3 lines
Quoted 3 lines
Quoted 2 lines

Of course apt uses dpkg to install a package as well, but it refuses to download the package from a repo when it's not signed.

Perhaps the difference is the packages are local and not remote?

No, for generating the rootfs it appears the sources list file has:

deb [trusted=yes] file:/path.../oe-rootfs-repo/edison/ ./

So apparently has been taken care by disabling the signing requirement. [trusted=yes] is part of that workaround.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

I guess we must configure apt to override that during the rootfs process and likely an end user with a remote feed could do the same, possibly with a warning
from apt?

Yes. But it is a pain to google to find how to do that.

Quoted 18 lines
Quoted 18 lines
Quoted 18 lines

 I believe there is no issue during rootfs generation.

Quoted 9 lines

I'm also worried that there isn't any automated testing of this change. The reason I worry is that since we don't show any testing failures right now, there
is clearly a hole in our automated testing coverage and there is no guarantee
that this feature will keep working. It is these smaller corner case issues which tend to make or break the project's experience as if a feature is present,
people expect it to work. Can we improve the testing situation?

It doesn't seem to be a particularly volatile area in the code. I refreshed Xavier's patches for Gatesgarth, and am actively using unchanged patch on Honisiter.
I don't know how the automated testing is working but I guess for RPM a repo is generated using a small layer? And then tested on a qemu running the rootfs?
Should be almost same for deb/apt, maybe could be modified from rpm test?

Quoted 5 lines
Quoted 4 lines

I think the rpm test is test_testimage_dnf in meta/lib/oeqa/selftest/cases/runtime_test.py. You'd run it with:

oe-selftest -r runtime_test.TestImage.test_testimage_dnf

I'll have a look.

Pfff. These tests are O(1) more then the signing code they test. I'm trying this original test (on rpm) now to see if I can get that to run and understand how it works.

Am I right that meta/lib/oeqa/selftest/cases/signing.py tests the actual package feed signing?

Quoted 30 lines
Quoted 15 lines
Quoted 5 lines

Point is: currently deb is documented as a feasible package format to generate a repo. But it really is not without these signing patches. So we could either deprecate deb's (no, no please don't) or fix it.
These patches fix it. With or without automated testing, it is already better then the current situation.

I'm not sure it is. The project gains yet another set of config options which we don't have tests for and the maintainer stress levels in trying to keep it all working just get worse.

We have a huge number of ways people can configure the builds. The only way the project manages to keep all of this working is through automated testing, there is simply no other way to do it. I appreciate adding tests is a pain and nobody likes doing it. It does however let the project stay functional for everyone's diverse use cases.

Quoted 4 lines

There are all kinds of patches I could take because the improve some corner case. In most cases we don't know what else they might break or whether that code continues to be used or stays working. I therefore do really want tests for new configurations.

It's not an improvement but a fix for an existing config. Even if it would break in the future it can't be worse then it is now.

Quoted 2 lines

Who should write those tests? If you don't/can't as some actively using the feature, it means someone else has to, but who?

I do understand.

Quoted 7 lines

I've not decided what to do with the patches to be honest but the merge and not bother with tests argument doesn't sit well with me.

Cheers,

Richard

#8 lore raw
Quoted 23 lines · Ferry Toth

Op 24-03-2022 om 16:36 schreef Ferry Toth:

Quoted 21 lines

Op 24-03-2022 om 13:03 schreef Richard Purdie:

Quoted 20 lines · Ferry Toth
Quoted 19 lines
Quoted 19 lines · Ferry Toth
Quoted 6 lines

I forgot to add a cover letter, sorry for that. The 2 patches  together implement DEB repository signing.

This is necessary since Gatesgarth |apt| (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.

Quoted 7 lines

It is possible to override this behavior |but||| is more work then to enable signed DEB repositories. These patches makes DEB a first class citizen as IPK and RPM.

Patches have been in use in meta-intel-edison since Gatesgarth, see
https://edison-fw.github.io/meta-intel-edison/5.0-Creating-a-deb-
repository.html\

What puzzles me is that we can build root filesystems using apt, we test this on
the autobuilder. Saying repositories are broken since gatesgarth therefore seems
confusing in the commit message.

Quoted 6 lines

The Development Task manual in 3.22.4.3.3 names it a repository, as I believe is common for a distributions server holding pre-built packages. But I could change the name to package feed if that would be better.

What I meant to say is that apt requires signed package feeds since 1.8.2 and disabling that is a workaround.

I mean to distinguish between the use of repositories at rootfs creation time compared to repositories used later to update an image. Since I know repositories are used to build the rootfs, the commit message did confuse me at first since rootfs construction works.

Quoted 9 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

Good question. When I (meta-intel-edison) build the rootfs using DEB's it just works.
Could it be that during rootfs build dpkg is used and not apt? I think I have seen that in the logs.

It definitely uses apt.

Checking logs I see you are correct.

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

Of course apt uses dpkg to install a package as well, but it refuses to download the package from a repo when it's not signed.

Quoted 8 lines
Quoted 1 line
Quoted 1 line

Perhaps the difference is the packages are local and not remote?

No, for generating the rootfs it appears the sources list file has:

deb [trusted=yes] file:/path.../oe-rootfs-repo/edison/ ./

So apparently has been taken care by disabling the signing requirement. [trusted=yes] is part of that workaround.

That does give another way to work around this then...

Quoted 27 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

I guess we must configure apt to override that during the rootfs process and likely an end user with a remote feed could do the same, possibly with a warning
from apt?

Yes. But it is a pain to google to find how to do that.

Quoted 21 lines
Quoted 21 lines
Quoted 11 lines

 I believe there is no issue during rootfs generation.

Quoted 9 lines

I'm also worried that there isn't any automated testing of this change. The reason I worry is that since we don't show any testing failures right now, there
is clearly a hole in our automated testing coverage and there is no guarantee
that this feature will keep working. It is these smaller corner case issues which tend to make or break the project's experience as if a feature is present,
people expect it to work. Can we improve the testing situation?

Quoted 7 lines

It doesn't seem to be a particularly volatile area in the code. I refreshed Xavier's patches for Gatesgarth, and am actively using unchanged patch on Honisiter.
I don't know how the automated testing is working but I guess for RPM a repo is generated using a small layer? And then tested on a qemu running the rootfs?
Should be almost same for deb/apt, maybe could be modified from rpm test?

I think the rpm test is test_testimage_dnf in meta/lib/oeqa/selftest/cases/runtime_test.py. You'd run it with:

Quoted 6 lines
Quoted 2 lines
Quoted 1 line

oe-selftest -r runtime_test.TestImage.test_testimage_dnf

I'll have a look.

Pfff. These tests are O(1) more then the signing code they test. I'm trying this original test (on rpm) now to see if I can get that to run and understand how it works.

What you're asking to work is a complex scenario so the test isn't going to be simple.

FWIW there is also oeqa/runtime/cases/apt.py. That test checks if a package can be installed from a remote package feed (no signing). It is doing:

echo deb [ allow-insecure=yes ] %s ./ > sources.list

to work around the signing issue.

Quoted 2 lines

Am I right that meta/lib/oeqa/selftest/cases/signing.py tests the actual package feed signing?

No. That tests that signed packages in rpm works rather than a signed remote repository which is different.

Cheers,

Richard

author
#9 lore raw

Hi,

Op 24-03-2022 om 23:20 schreef Richard Purdie:

Quoted 16 lines · Ferry Toth
Quoted 15 lines

Op 24-03-2022 om 16:36 schreef Ferry Toth:

Quoted 14 lines

Op 24-03-2022 om 13:03 schreef Richard Purdie:

Quoted 13 lines · Ferry Toth
Quoted 12 lines
Quoted 12 lines · Ferry Toth
Quoted 10 lines

I forgot to add a cover letter, sorry for that. The 2 patches  together implement DEB repository signing.
This is necessary since Gatesgarth |apt| (1.8.2) has become more strict and doesn’t allow unsigned repositories by default.
It is possible to override this behavior |but||| is more work then to enable signed DEB repositories. These patches makes DEB a first class citizen as IPK and RPM.
Patches have been in use in meta-intel-edison since Gatesgarth, see
https://edison-fw.github.io/meta-intel-edison/5.0-Creating-a-deb-
repository.html\

<snip>

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

I think the rpm test is test_testimage_dnf in meta/lib/oeqa/selftest/cases/runtime_test.py. You'd run it with:
oe-selftest -r runtime_test.TestImage.test_testimage_dnf

I'll have a look.

I got the rpm test to run and with the help of Alexander I can restart that test to complete in about 6 minutes, which is good.

Right now, I have the modified the recipe so the test image is built from deb packages and then runs on qemu.

So next I need the workaround for apt disabled, auto install the gpg key on the testimage, run apt and parse the results.

I'm not really fluent with python, but might be able to get this done. If Alexander would be willing to give me a few pointers if I get stuck?

Let me play with this a few days (evenings) and then I'll post a WIP patch.

Quoted 5 lines

Cheers,

Richard

author
#10 lore raw
In reply to Ferry Toth

Hi,

Op 24-03-2022 om 23:11 schreef Ferry Toth:
....

Quoted 12 lines

It doesn't seem to be a particularly volatile area in the code. I refreshed

Quoted 11 lines
Quoted 10 lines
Quoted 6 lines

Xavier's patches for Gatesgarth, and am actively using unchanged patch on Honisiter.
I don't know how the automated testing is working but I guess for RPM a repo is generated using a small layer? And then tested on a qemu running the rootfs?
Should be almost same for deb/apt, maybe could be modified from rpm test?

I think the rpm test is test_testimage_dnf in meta/lib/oeqa/selftest/cases/runtime_test.py. You'd run it with:

oe-selftest -r runtime_test.TestImage.test_testimage_dnf

I'll have a look.

Grumble. I ran this test but it failed due to needing to run sudo to setup a tap interface (I ran it under screen).

Is there a way to continue this test without first clearing the build-st directory? It take about 2 hours to get to this failure point.

Thanks,

Ferry

#11 lore raw

You can copy the sstate cache out of build-st somewhere where it will be reused (set up local.conf accordingly).

Alex

Quoted 39 lines · Ferry Toth

Hi,

Op 24-03-2022 om 23:11 schreef Ferry Toth:
....

It doesn't seem to be a particularly volatile area in the code. I refreshed

Xavier's patches for Gatesgarth, and am actively using unchanged patch on Honisiter.
I don't know how the automated testing is working but I guess for RPM a repo is generated using a small layer? And then tested on a qemu running the rootfs?
Should be almost same for deb/apt, maybe could be modified from rpm test?

I think the rpm test is test_testimage_dnf in meta/lib/oeqa/selftest/cases/runtime_test.py. You'd run it with:

oe-selftest -r runtime_test.TestImage.test_testimage_dnf

I'll have a look.

Grumble. I ran this test but it failed due to needing to run sudo to setup a tap interface (I ran it under screen).

Is there a way to continue this test without first clearing the build-st directory? It take about 2 hours to get to this failure point.

Thanks,

Ferry

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#163636): https://lists.openembedded.org/g/openembedded-core/message/163636
Mute This Topic: https://lists.openembedded.org/mt/89962631/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-