debian packaging: set md5sums for conffiles
PATCH 1/1 debian packaging: set md5sums for conffiles
Currently in an image created with debian package management, dpkg is missing the md5sums in /var/lib/dpkg/status for files designated as conffiles. They are just listed as "newconffile", which is an intermediate state when dpkg installs a package with an conf file. Also, when the device first boots the script /usr/sbin/run-postinsts does not cause /var/lib/dpkg/status to update those hashes either.
The problem here is that when using a package feed, all the conf files are listed as "newconffile", which will indicate to dpkg to replace them all on the first update. dpkg will not follow the rules for confold, confnew, confdef.
--- meta/lib/oe/package_manager/deb/__init__.py | 42 +++++++++++++++++++++ meta/lib/oe/package_manager/deb/rootfs.py | 2 + 2 files changed, 44 insertions(+)
meta/lib/oe/package_manager/deb/__init__.py +42 -0
@@ -3,6 +3,7 @@ # import re +import glob import subprocess from oe.package_manager import *
@@ -216,6 +217,47 @@ class DpkgPM(OpkgDpkgPM): os.rename(status_file + ".tmp", status_file) + def set_conffile_hashes(self): + """ + This function will populate the md5sums for all the conffiles listed in /var/lib/dpkg/status + """ + status_file = self.target_rootfs + "/var/lib/dpkg/status" + + with open(status_file, "r") as sf: + with open(status_file + ".tmp", "w+") as tmp_sf: + status = sf.read() + + conffiles = glob.glob(self.target_rootfs + "/var/lib/dpkg/info/*.conffiles") + for cf in conffiles: + fname = os.path.basename(cf) + pkg = fname.split('.')[0] + + bb.note("Populating md5sums for pkg %s" % pkg) + conffiles = "%s/var/lib/dpkg/info/%s.conffiles" % (self.target_rootfs, pkg) + md5sums = "%s/var/lib/dpkg/info/%s.md5sums" % (self.target_rootfs, pkg) + if os.path.exists(conffiles) and os.path.exists(md5sums): + conf_hashes = {} + with open(md5sums, "r") as md5s: + for line in md5s.read().split('\n'): + m = re.match(r"^([a-f0-9]+)\s+(\S+)", line) + if m: + hash_ = m.group(1) + fpath = m.group(2) + if not fpath.startswith('/'): + fpath = '/' + fpath + conf_hashes[fpath] = hash_ + + with open(conffiles, "r") as cf: + for fpath in cf.read().split('\n'): + hash_ = conf_hashes.get(fpath, None) + if hash_: + status = re.sub("%s newconffile" % fpath, "%s %s" % (fpath, hash_), status) + + + tmp_sf.write(status) + + os.rename(status_file + ".tmp", status_file) + def run_pre_post_installs(self, package_name=None): """ Run the pre/post installs for package "package_name". If package_name is
meta/lib/oe/package_manager/deb/rootfs.py +2 -0
@@ -186,6 +186,8 @@ class PkgRootfs(DpkgOpkgRootfs): execute_pre_post_process(self.d, deb_post_process_cmds) + self.pm.set_conffile_hashes() + if self.progress_reporter: self.progress_reporter.next_stage()
Signature · 1 line
2.25.1
Hello Rusty,
Quoted 9 lines · On Tue, 26 Jul 2022 15:13:28 -0600 "Rusty Howell" <[email protected]> wrote:
Currently in an image created with debian package management, dpkg is missing the md5sums in
/var/lib/dpkg/statusfor files designated as conffiles. They are just listed as "newconffile", which is an intermediate state when dpkg installs a package with an conf file. Also, when the device first boots the script/usr/sbin/run-postinstsdoes not cause/var/lib/dpkg/statusto update those hashes either.The problem here is that when using a package feed, all the conf files are listed as "newconffile", which will indicate to dpkg to replace them all on the first update. dpkg will not follow the rules for confold, confnew, confdef.
This patch does not apply on the current master branch of oe-core. Maybe you wrote it on an older commit or on a branch (e.g. kirkstone)?
Also we require you so sign-off the patches you send on the mailing list ('git commit -s' does that automatically).
Best regards.
Signature · 3 lines
Luca Ceresoli, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
Also, I'm not sure I understand why this needs to be fixed with custom code. Isn't this something that dpkg/apt take care of themselves?
Alex
Quoted 36 lines · Luca Ceresoli via lists.openembedded.org
Hello Rusty,
Quoted 9 lines · On Tue, 26 Jul 2022 15:13:28 -0600 "Rusty Howell" <[email protected]> wrote:
Currently in an image created with debian package management, dpkg is missing the md5sums in
/var/lib/dpkg/statusfor files designated as conffiles. They are just listed as "newconffile", which is an intermediate state when dpkg installs a package with an conf file. Also, when the device first boots the script/usr/sbin/run-postinstsdoes not cause/var/lib/dpkg/statusto update those hashes either.The problem here is that when using a package feed, all the conf files are listed as "newconffile", which will indicate to dpkg to replace them all on the first update. dpkg will not follow the rules for confold, confnew, confdef.
This patch does not apply on the current master branch of oe-core. Maybe you wrote it on an older commit or on a branch (e.g. kirkstone)?
Also we require you so sign-off the patches you send on the mailing list ('git commit -s' does that automatically).
Best regards.
--
Luca Ceresoli, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#168552): https://lists.openembedded.org/g/openembedded-core/message/168552
Mute This Topic: https://lists.openembedded.org/mt/92637328/1686489
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Quoted 5 lines · Alexander Kanavin via lists.openembedded.org
Also, I'm not sure I understand why this needs to be fixed with custom code. Isn't this something that dpkg/apt take care of themselves?
Yes, I suspect this is a problem with how we’re generating the DEBIAN/.
Ross
So we’re definitely not generating the DEBIAN/md5sums files that dpkg-deb likes to have. However, this file in proper Debian packages doesn’t contain hashes for the conffiles, so I can only imagine that dpkg generates them on install, which leads to the question of where that is meant to happen.
Ross
Quoted 1 line
So we’re definitely not generating the DEBIAN/md5sums files that dpkg-deb likes to have. However, this file in proper Debian packages doesn’t contain hashes for the conffiles, so I can only imagine that dpkg generates them on install, which leads to the question of where that is meant to happen.
dpkg —configure does this, but we’re neutering it by renaming all foo.dpkg-new files to foo in the deb install() function.
If we’re going to rename foo.dpkg-new to foo, then that is where we go and update the status file.
Alternatively, we don’t do that, and every package needs to run its configure step on first boot.
Ross
Thanks for the quick responses. It took us a while to figure out why we
were seeing weird behaviour with our products, and we ultimately figured
out the conffiles are being replaced during the update. I am working off
of Hardknott currently, so that's my mistake not switching to master. And I guess I misunderstood who is allowed to "sign off" on patches. I assumed I couldn't since I'm not a regular. But in any case, it looks like Ross Burton may be on to a better solution than what I have to fix the issue.
Rusty
Quoted 27 lines · Ross Burton
Quoted 1 line
So we’re definitely not generating the DEBIAN/md5sums files that
dpkg-deb likes to have. However, this file in proper Debian packages doesn’t contain hashes for the conffiles, so I can only imagine that dpkg generates them on install, which leads to the question of where that is meant to happen.
dpkg —configure does this, but we’re neutering it by renaming all foo.dpkg-new files to foo in the deb
install()function.If we’re going to rename foo.dpkg-new to foo, then that is where we go and update the status file.
Alternatively, we don’t do that, and every package needs to run its configure step on first boot.
Ross
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#168563):
https://lists.openembedded.org/g/openembedded-core/message/168563
Mute This Topic: https://lists.openembedded.org/mt/92637328/4800265
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ [email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
I think Ross is merely helping you to reach the correct solution, but everyone would appreciate if you write and submit the actual fix :-)
Alex
Quoted 24 lines · Rusty Howell
Thanks for the quick responses. It took us a while to figure out why we were seeing weird behaviour with our products, and we ultimately figured out the conffiles are being replaced during the update. I am working off of Hardknott currently, so that's my mistake not switching to master. And I guess I misunderstood who is allowed to "sign off" on patches. I assumed I couldn't since I'm not a regular. But in any case, it looks like Ross Burton may be on to a better solution than what I have to fix the issue.
Rusty
Quoted 18 lines · Ross Burton
Quoted 2 lines
So we’re definitely not generating the DEBIAN/md5sums files that dpkg-deb likes to have. However, this file in proper Debian packages doesn’t contain hashes for the conffiles, so I can only imagine that dpkg generates them on install, which leads to the question of where that is meant to happen.
dpkg —configure does this, but we’re neutering it by renaming all foo.dpkg-new files to foo in the deb
install()function.If we’re going to rename foo.dpkg-new to foo, then that is where we go and update the status file.
Alternatively, we don’t do that, and every package needs to run its configure step on first boot.
Ross
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#168563): https://lists.openembedded.org/g/openembedded-core/message/168563
Mute This Topic: https://lists.openembedded.org/mt/92637328/4800265
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
Will do.
Quoted 46 lines · Alexander Kanavin
I think Ross is merely helping you to reach the correct solution, but everyone would appreciate if you write and submit the actual fix :-)
Alex
Quoted 2 lines · Rusty Howell
Thanks for the quick responses. It took us a while to figure out why
we were seeing weird behaviour with our products, and we ultimately figured
out the conffiles are being replaced during the update. I am working off
of Hardknott currently, so that's my mistake not switching to master. And I guess I misunderstood who is allowed to "sign off" on patches. I assumed I couldn't since I'm not a regular. But in any case, it looks like Ross Burton may be on to a better solution than what I have to fix the issue.Quoted 6 lines
Rusty
Quoted 2 lines · Ross Burton
Quoted 2 lines
So we’re definitely not generating the DEBIAN/md5sums files that
dpkg-deb likes to have. However, this file in proper Debian packages doesn’t contain hashes for the conffiles, so I can only imagine that dpkg generates them on install, which leads to the question of where that is meant to happen.
Quoted 2 lines
Quoted 2 lines
dpkg —configure does this, but we’re neutering it by renaming all
foo.dpkg-new files to foo in the deb
install()function.Quoted 2 lines
Quoted 2 lines
If we’re going to rename foo.dpkg-new to foo, then that is where we go
and update the status file.
Quoted 2 lines
Quoted 2 lines
Alternatively, we don’t do that, and every package needs to run its
configure step on first boot.
Quoted 5 lines
Quoted 5 lines
Ross
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group. View/Reply Online (#168563):https://lists.openembedded.org/g/openembedded-core/message/168563
Quoted 3 lines
Quoted 3 lines
Mute This Topic: https://lists.openembedded.org/mt/92637328/4800265
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [Quoted 2 lines
Quoted 2 lines
-=-=-=-=-=-=-=-=-=-=-=-