Ext3 rootfs Corrupted during do_rootfs
Hello,
I think I may have encountered the same issue as reported originally on the meta freescale list...
https://lists.yoctoproject.org/pipermail/meta-freescale/2014-February/006928.html
...and...
https://lists.yoctoproject.org/pipermail/meta-freescale/2014-February/007086.html
In my case, after failed boot on Wandboard I notice that the ext3 filesystem was corrupt. No 'boot' dir and no 'lib' dir but some of those files were in the root dir. I have sometimes observed files in place of what should be directory names.
I took the sdcard and 'dd' the ext3 image over the rootfs partition (leaving the u-boot partition alone) and found the end result to be the same. This led me to dig into the ext3 filesystem generation rather than the final assembly into an sdcard image. (This also is why I report it on yocto list rather than freescale list).
I think something is going wrong while oe_mkext234fs() inside image_types.bbclass is running the populate-extfs.sh script.
Below are some error message from the log.do_rootfs.3618 output.
At line# 42993... first suspicious looking output
-----------------
debugfs: write "/home/darcy/workspace/epsilon/imt-repo-gerrit-1/build/tmp/work/omg_epsilon_wbqfs-poky-linux-gnueabi/omg-image-full/1.0-r0/rootfs/usr/bin/mc" "mc" Allocated inode: 6656copy_file: Could not allocate block in ext2 filesystem
debugfs: sif "mc" mode 0x81ed
debugfs: sif "mc" uid 0
debugfs: sif "mc" gid 0
And every copy attempt downstream was similarly affected.
Some other similar errors...
debugfs: write "/home/darcy/workspace/epsilon/imt-repo-gerrit-1/build/tmp/work/omg_epsilon_wbqfs-poky-linux-gnueabi/omg-image-full/1.0-r0/rootfs/usr/bin/git-receive-pack" "git-receive-pack" Allocated inode: 6701
write: Could not allocate block in ext2 filesystem while expanding directory
debugfs: sif "git-receive-pack" mode 0x81ed
git-receive-pack: File not found by ext2_lookup
debugfs: sif "git-receive-pack" uid 0
git-receive-pack: File not found by ext2_lookup
debugfs: sif "git-receive-pack" gid 0
git-receive-pack: File not found by ext2_lookup
debugfs: sif "head" mode 0xa1ff
debugfs: sif "head" uid 0
debugfs: sif "head" gid 0
debugfs: symlink "busclk" "omgtool"
symlink: Could not allocate block in ext2 filesystem while expanding directory
debugfs: sif "busclk" mode 0xa1ff
busclk: File not found by ext2_lookup
debugfs: sif "busclk" uid 0
busclk: File not found by ext2_lookup
debugfs: sif "busclk" gid 0
busclk: File not found by ext2_lookup
debugfs: write "/home/darcy/workspace/epsilon/imt-repo-gerrit-1/build/tmp/work/omg_epsilon_wbqfs-poky-linux-gnueabi/omg-image-full/1.0-r0/rootfs/usr/bin/numfmt" "numfmt" Allocated inode: 6702
write: Could not allocate block in ext2 filesystem while expanding directory
debugfs: sif "numfmt" mode 0x81ed
numfmt: File not found by ext2_lookup
debugfs: sif "numfmt" uid 0
numfmt: File not found by ext2_lookup
debugfs: sif "numfmt" gid 0
numfmt: File not found by ext2_lookup
debugfs: symlink "killall" "/usr/bin/killall.psmisc"
symlink: Could not allocate block in ext2 filesystem while expanding directory
debugfs: sif "killall" mode 0xa1ff
killall: File not found by ext2_lookup
debugfs: sif "killall" uid 0
killall: File not found by ext2_lookup
debugfs: sif "killall" gid 0
killall: File not found by ext2_lookup
... and a lot more of the same
This rootfs is 369M in size.
Attached is full log file (tarballed).
Do you think that the rootfs size estimation algorithm is underestimating the rootfs size and that this is just what happens when the space that it allocates gets all filled up?
I am using 'dora' branch building a custom image (quite large with java and all sorts of networking stuff in it). The target is i.MX6 based wandboard quad (although I encounter the same when building the same distro / image for imxqsabresd).
Signature · 6 lines
Regards, Darcy http://xstreamworship.com [P1]
Quoted 7 lines · Darcy Watkins
Hello,
I think I may have encountered the same issue as reported originally on the meta freescale list...
https://lists.yoctoproject.org/pipermail/meta-freescale/2014-February/006928.html
...and...
https://lists.yoctoproject.org/pipermail/meta-freescale/2014-February/007086.html
--snip!--
Found a workaround! I set the IMAGE_ROOTFS_EXTRA_SPACE to a nice 128M margin and got everything working OK.
So it would appear that the rootfs size estimation algorithm underestimates the needed size. Perhaps using 'du' on the build host results in variances between how it lays out on the host versus the filesystem being constructed for the target.
The algorithm includes a margin for error, so we run into trouble when the variances accumulate to exceed this margin.
Perhaps worse than any of this is that the populate-extfs.sh script didn't catch the failures to exit with an error code (i.e. to break the build rather than leave a corrupt rootfs).
Hopefully this info helps.
I have observed symptoms of the filesystem corrupted this way as any of:
- kernel boots but unable to complete init because things are missing
- numerous error messages about commands missing during startup
- a seemingly random file at a path that should be a directory
- files in wrong location, usually closer to the root directory
- broken symlinks
- commands / programs / libraries missing
Signature · 6 lines
Regards, Darcy http://xstreamworship.com [P1]