lib/oe/rootfs: Fix DEBUGFS generation when using opkg
PATCH 1/1 lib/oe/rootfs: Fix DEBUGFS generation when using opkg
When enabling extra DEBUGFS image generation with opkg, errors are seen like:
ERROR: core-image-minimal-1.0-r0 do_rootfs: Cannot get the installed packages list. Command '/media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/bin/opkg -f /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/opkg.conf -o /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs --force_postinstall --prefer-arch-to-version status' returned 0 and stderr:
Collected errors:
* file_md5sum_alloc: Failed to open file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/syslog-startup.conf.busybox: No such file or directory.
* file_md5sum_alloc: Failed to open file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/fstab: No such file or directory.
basically for all CONFFILES in the image. This is due to the file rearranging the rootfs generation code does. If we preserve the /etc directory, the avoids the problem.
We need to tell copyfile to preserve symlinks since some are present in /etc.
[YOCTO #9490]
meta/lib/oe/rootfs.py +2 -2
@@ -122,7 +122,7 @@ class Rootfs(object): bb.note(" Copying back package database...") for dir in dirs: bb.utils.mkdirhier(self.image_rootfs + os.path.dirname(dir)) - shutil.copytree(self.image_rootfs + '-orig' + dir, self.image_rootfs + dir) + shutil.copytree(self.image_rootfs + '-orig' + dir, self.image_rootfs + dir, symlinks=True) cpath = oe.cachedpath.CachedPath() # Copy files located in /usr/lib/debug or /usr/src/debug
@@ -907,7 +907,7 @@ class OpkgRootfs(DpkgOpkgRootfs): self.pm.install_complementary() - self._setup_dbg_rootfs(['/var/lib/opkg']) + self._setup_dbg_rootfs(['/etc', '/var/lib/opkg']) execute_pre_post_process(self.d, opkg_post_process_cmds)
Hi,
Quoted 7 lines · Richard Purdie
When enabling extra DEBUGFS image generation with opkg, errors are seen like:
ERROR: core-image-minimal-1.0-r0
do_rootfs: Cannot get the installed packages list. Command '/media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/bin/opkg -f /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/opkg.conf -o /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs --force_postinstall --prefer-arch-to-version status' returned 0 and stderr:
Collected errors:
*file_md5sum_alloc: Failed to open file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/syslog-startup.conf.busybox: No such file or directory.
*file_md5sum_alloc: Failed to open file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/fstab: No such file or directory.
Quoted 5 lines
basically for all CONFFILES in the image. This is due to the file rearranging the rootfs generation code does. If we preserve the /etc directory, the avoids the problem.
We need to tell copyfile to preserve symlinks since some are present in /etc.
Thanks for the patch. It fixes the issue on e.g. core-image-minimal.
Unfortunately, on more complex images, we'll still fail with e.g.:
ERROR: core-image-minimal-1.0-r0 do_rootfs: Cannot get the installed packages list. Command '/mnt/cs-builds/anders/poky-play/build/tmp-glibc/sysroots/x86_64-linux/usr/bin/opkg -f /mnt/cs-builds/anders/poky-play/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/opkg.conf -o /mnt/cs-builds/anders/poky-play/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs --force_postinstall --prefer-arch-to-version status' returned 0 and stderr:
Collected errors:
* file_md5sum_alloc: Failed to open file /mnt/cs-builds/anders/poky-play/build/tmp-glibc/work/qemux86-oe-linux/core-image-minimal/1.0-r0/rootfs/usr/lib/ssl/openssl.cnf: No such file or directory.
I'd guess that all packages that have CONFFILES outside of /etc will suffer from this...
I've updated #9490 with this info.
Cheers,
Anders
Signature · 2 lines
Anders Darander, Senior System Architect ChargeStorm AB / eStorm AB
Quoted 9 lines · Richard Purdie
When enabling extra DEBUGFS image generation with opkg, errors are seen like:
ERROR: core-image-minimal-1.0-r0
do_rootfs: Cannot get the installed packages list. Command '/media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/bin/opkg -f /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/opkg.conf -o /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs --force_postinstall --prefer-arch-to-version status' returned 0 and stderr:
Collected errors:
*file_md5sum_alloc: Failed to open
file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/syslog-startup.conf.busybox: No such file or directory.
*file_md5sum_alloc: Failed to open
file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/fstab: No such file or directory.
Quoted 4 lines
basically for all CONFFILES in the image. This is due to the file rearranging
the rootfs generation code does. If we preserve the /etc directory, the avoids the problem.
It avoids the problem, but it does not address the root cause (IMHO). Should opkg really complain about missing configuration files? It is perhaps an edge case, but besides editing a configuration file *removing* it entirely may also be a valid user modification.
The usage of opkg here is during image building, but the operation itself (opkg status) is not specific to image creation and thus should be able to handle arbitrary states of /etc and the config files in general.
Signature · 6 lines
Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.
Quoted 20 lines · Patrick Ohly
Quoted 9 lines · Richard Purdie
When enabling extra DEBUGFS image generation with opkg, errors are seen like:
ERROR: core-image-minimal-1.0-r0
do_rootfs: Cannot get the installed packages list. Command '/media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/bin/opkg -f /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/opkg.conf -o /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs --force_postinstall --prefer-arch-to-version status' returned 0 and stderr:
Collected errors:
*file_md5sum_alloc: Failed to open
file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/syslog-startup.conf.busybox: No such file or directory.
*file_md5sum_alloc: Failed to open
file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/fstab: No such file or directory.Quoted 4 lines
basically for all CONFFILES in the image. This is due to the file rearranging
the rootfs generation code does. If we preserve the /etc directory, the avoids the problem.It avoids the problem, but it does not address the root cause (IMHO). Should opkg really complain about missing configuration files? It is perhaps an edge case, but besides editing a configuration file *removing* it entirely may also be a valid user modification.
Quoted 4 lines
The usage of opkg here is during image building, but the operation itself (opkg status) is not specific to image creation and thus should be able to handle arbitrary states of /etc and the config files in general.
Opkg status is supposed to give you the state of all your installed packages. Currently is pretty rough data: it dumps into stdout the internal metadata that tracks package info and promotes to error any inconsistency, like conffiles not being present.
Looking at pacakge_manager.py, opkg status is being called on OpkgPkgsList. Seems to me that if the purpose is to get all installed packages, then "opkg list-installed" is the correct command. Looking at dpkg (DpkgPkgsList), "dpkg-query -W" is being called, which I think maps to opkg list-installed, not to opkg status.
Signature · 3 lines
Cheers, Alejandro
Quoted 27 lines · Alejandro del Castillo
Quoted 20 lines · Patrick Ohly
Quoted 9 lines · Richard Purdie
When enabling extra DEBUGFS image generation with opkg, errors are seen like:
ERROR: core-image-minimal-1.0-r0
do_rootfs: Cannot get the installed packages list. Command '/media/build1/poky/build/tmp/sysroots/x86_64-linux/usr/bin/opkg -f /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/opkg.conf -o /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs --force_postinstall --prefer-arch-to-version status' returned 0 and stderr:
Collected errors:
*file_md5sum_alloc: Failed to open
file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/syslog-startup.conf.busybox: No such file or directory.
*file_md5sum_alloc: Failed to open
file /media/build1/poky/build/tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs/etc/fstab: No such file or directory.Quoted 4 lines
basically for all CONFFILES in the image. This is due to the file rearranging
the rootfs generation code does. If we preserve the /etc directory, the avoids the problem.It avoids the problem, but it does not address the root cause (IMHO). Should opkg really complain about missing configuration files? It is perhaps an edge case, but besides editing a configuration file *removing* it entirely may also be a valid user modification.
Quoted 4 lines
The usage of opkg here is during image building, but the operation itself (opkg status) is not specific to image creation and thus should be able to handle arbitrary states of /etc and the config files in general.
Quoted 4 lines
Opkg status is supposed to give you the state of all your installed packages. Currently is pretty rough data: it dumps into stdout the internal metadata that tracks package info and promotes to error any inconsistency, like conffiles not being present.
But how does opkg know that "config file not present" is an error? It depends on the the semantic of the file and thus the package providing the file. It's not declared explicitly, so it is a fairly arbitrary judgment call to treat "missing" as error.
Quoted 5 lines
Looking at
pacakge_manager.py, opkg status is being called on OpkgPkgsList. Seems to me that if the purpose is to get all installed packages, then "opkg list-installed" is the correct command. Looking at dpkg (DpkgPkgsList), "dpkg-query -W" is being called, which I think maps to opkg list-installed, not to opkg status.
That indeed seems better: output should be smaller, faster to run and it avoids the ambiguity about missing config files.
Signature · 6 lines
Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter.
Quoted 4 lines · Patrick Ohly
But how does opkg know that "config file not present" is an error? It depends on the the semantic of the file and thus the package providing the file. It's not declared explicitly, so it is a fairly arbitrary judgment call to treat "missing" as error.
I agree. What I meant when I said that "opkg status" output is pretty rough is that it is making blanket assumptions, and dumping unprocessed metadata. Dpkg -V analyses the md5sum of each file to determine if it changed, and reports changed files. While opkg currently only stores the md5sum of conffiles, I think opkg status should behave similarly to dpkg -V (for conffiles, at least).
Do you mind filing a bug ticket on bugzilla for opkg?
Signature · 3 lines
Cheers, Alejandro