Fix bootimg bug 2138 for 1.1.2
cover Fix bootimg bug 2138 for 1.1.2
Backport the following patches to edison. The backport involves massaging this minimal set of patches to apply to bootimg.bbclass without pulling in the refactoring or EFI support added in master about the same time.
These patches have been tested by building atom-pc core-image-minimal with edison, a dosfsck and mount test on resulting image, and booting on the n450.
The following changes since commit 0fbd6a161576b2cafa8583adde0ffb15347c884a:
poky.conf: Bumping SKD_VERSION (2012-03-14 15:49:33 -0700)
are available in the git repository at:
git://git.pokylinux.org/poky-contrib dvhart/edison/bug/2138
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dvhart/edison/bug/2138
Darren Hart (4):
bootimg: Use mcopy to construct the hddimg
bootimg: Account for FAT filesystem overhead in image size
bootimg: Fix a math thinko in the block count calculation
bootimg: Do not force FAT32 on all images, it violates the FAT
specification
meta/classes/bootimg.bbclass | 53 ++++++++++++++++++++++++++++++++++++----- 1 files changed, 46 insertions(+), 7 deletions(-)
Signature · 1 line
1.7.5.4
PATCH 1/4 bootimg: Use mcopy to construct the hddimg
Fixes [YOCTO 2138]
The initial directory support (-d) added to mkdosfs has proven to be incomplete and non-compliant with FAT. Rather than continue to maintain this feature and work around the various issues, we can use mcopy to construct the image.
bootimg.bbclass already depends on mtools-native (although it may not have needed to previously). No new dependencies are introduced. The image created passes dosfsck cleanly. Remove the call to dosfsck.
mcopy reported an error with the image we were creating:
Total number of sectors (107574) not a multiple of sectors per track (32)!
Add some logic to ensure the total sector count is an integral number of sectors per track, including forcing the logical sector size to 512 in the mkdosfs command.
The du -bks arguments are contradictory, -b is equivalent to "--apparent-size --block-size=1" and -k is --block-size=1K. If reordered, -kbs will report the disk usage in bytes insteadk of 1k blocks. Eliminate the ambiguity by using:
du --apparent-size -ks
(From OE-Core rev: 92d2ea1a306354c6565a1b05b51b5719e481840f)
Backported to poky edison by Darren Hart.
--- meta/classes/bootimg.bbclass | 22 +++++++++++++++------- 1 files changed, 15 insertions(+), 7 deletions(-)
meta/classes/bootimg.bbclass +15 -7
@@ -62,13 +62,21 @@ build_boot_bin() { install -m 444 ${STAGING_LIBDIR}/syslinux/ldlinux.sys ${HDDDIR}/ldlinux.sys - # Do a little math, bash style - #BLOCKS=`du -s ${HDDDIR} | cut -f 1` - BLOCKS=`du -bks ${HDDDIR} | cut -f 1` - SIZE=`expr $BLOCKS + ${BOOTIMG_EXTRA_SPACE}` - - mkdosfs -n ${BOOTIMG_VOLUME_ID} -d ${HDDDIR} \ - -C ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hddimg $SIZE + # Determine the 1024 byte block count for the final image. + BLOCKS=`du --apparent-size -ks ${HDDDIR} | cut -f 1` + SIZE=`expr $BLOCKS + ${BOOTIMG_EXTRA_SPACE}` + + # Ensure total sectors is an integral number of sectors per + # track or mcopy will complain. Sectors are 512 bytes, and and + # we generate images with 32 sectors per track. This calculation + # is done in blocks, which are twice the size of sectors, thus + # the 16 instead of 32. + SIZE=$(expr $SIZE + $(expr 16 - $(expr $SIZE % 16))) + + IMG=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hddimg + mkdosfs -n ${BOOTIMG_VOLUME_ID} -S 512 -C ${IMG} ${SIZE} + # Copy HDDDIR recursively into the image file directly + mcopy -i ${IMG} -s ${HDDDIR}/* ::/ syslinux ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hddimg chmod 644 ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hddimg
Signature · 1 line
1.7.5.4
PATCH 2/4 bootimg: Account for FAT filesystem overhead in image size
In reply to Darren HartFixes [YOCTO #1852]
The bootimg class wasn't accounting for non-trivial amount of space required by the directory entries and FATs for the FAT filesystem.
This patch attempts to make an accurate prediction of FAT overhead and adjusts the image size accordingly. It assumes no more than 16 directory entries per directory (which fit in a single sector). It also assumes 8.3 filenames. With the ceiling functions rounding up to full sectors and tracks, these assumptions seem reasonable.
In order to ensure the calculations are accurate, this patch forces the FAT size to 32, rather than allowing mkdosfs to automatically select 12, 16, or 32 depending on the image being built.
Tested by setting BOOTIMG_EXTRA_SPACE=0 and building core-image-minimal and core-image-sato for fri2-noemgd from meta-intel.
(From OE-Core rev: 68aa18609c10a3ae2f738930c933fa2a95ce8959)
Backported to edison by Darren Hart.
--- meta/classes/bootimg.bbclass | 45 +++++++++++++++++++++++++++++++++-------- 1 files changed, 36 insertions(+), 9 deletions(-)
meta/classes/bootimg.bbclass +36 -9
@@ -62,19 +62,46 @@ build_boot_bin() { install -m 444 ${STAGING_LIBDIR}/syslinux/ldlinux.sys ${HDDDIR}/ldlinux.sys - # Determine the 1024 byte block count for the final image. - BLOCKS=`du --apparent-size -ks ${HDDDIR} | cut -f 1` - SIZE=`expr $BLOCKS + ${BOOTIMG_EXTRA_SPACE}` + # Calculate the size required for the final image including the + # data and filesystem overhead. + # Sectors: 512 bytes + # Blocks: 1024 bytes + + # Determine the sector count just for the data + SECTORS=$(expr $(du --apparent-size -ks ${HDDDIR} | cut -f 1) \* 2) + + # Account for the filesystem overhead. This includes directory + # entries in the clusters as well as the FAT itself. + # Assumptions: + # < 16 entries per directory + # 8.3 filenames only + + # 32 bytes per dir entry + DIR_BYTES=$(expr $(find ${HDDDIR} | tail -n +2 | wc -l) \* 32) + # 32 bytes for every end-of-directory dir entry + DIR_BYTES=$(expr $DIR_BYTES + $(expr $(find ${HDDDIR} -type d | tail -n +2 | wc -l) \* 32)) + # 4 bytes per FAT entry per sector of data + FAT_BYTES=$(expr $SECTORS \* 4) + # 4 bytes per FAT entry per end-of-cluster list + FAT_BYTES=$(expr $FAT_BYTES + $(expr $(find ${HDDDIR} -type d | tail -n +2 | wc -l) \* 4)) + + # Use a ceiling function to determine FS overhead in sectors + DIR_SECTORS=$(expr $(expr $DIR_BYTES + 511) / 512) + # There are two FATs on the image + FAT_SECTORS=$(expr $(expr $(expr $FAT_BYTES + 511) / 512) \* 2) + SECTORS=$(expr $SECTORS + $(expr $DIR_SECTORS + $FAT_SECTORS)) + + # Determine the final size in blocks accounting for some padding + BLOCKS=$(expr $(expr $SECTORS \* 2) + ${BOOTIMG_EXTRA_SPACE}) # Ensure total sectors is an integral number of sectors per - # track or mcopy will complain. Sectors are 512 bytes, and and - # we generate images with 32 sectors per track. This calculation - # is done in blocks, which are twice the size of sectors, thus - # the 16 instead of 32. - SIZE=$(expr $SIZE + $(expr 16 - $(expr $SIZE % 16))) + # track or mcopy will complain. Sectors are 512 bytes, and we + # generate images with 32 sectors per track. This calculation is + # done in blocks, thus the mod by 16 instead of 32. + BLOCKS=$(expr $BLOCKS + $(expr 16 - $(expr $BLOCKS % 16))) IMG=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hddimg - mkdosfs -n ${BOOTIMG_VOLUME_ID} -S 512 -C ${IMG} ${SIZE} + mkdosfs -F 32 -n ${BOOTIMG_VOLUME_ID} -S 512 -C ${IMG} ${BLOCKS} # Copy HDDDIR recursively into the image file directly mcopy -i ${IMG} -s ${HDDDIR}/* ::/
Signature · 1 line
1.7.5.4
PATCH 3/4 bootimg: Fix a math thinko in the block count calculation
In reply to Darren HartFixes [YOCTO #1852] ... again.
The conversion from sectors to blocks was multiplying by 2 instead of dividing by 2. Blocks are 1024 bytes, sectors are 512 bytes. The result was images being much larger than intended.
(From OE-Core rev: b35384fa3ca96b31c63d764322215abced2066e4)
Backported to edison by Darren Hart.
--- meta/classes/bootimg.bbclass | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-)
meta/classes/bootimg.bbclass +2 -1
@@ -92,7 +92,8 @@ build_boot_bin() { SECTORS=$(expr $SECTORS + $(expr $DIR_SECTORS + $FAT_SECTORS)) # Determine the final size in blocks accounting for some padding - BLOCKS=$(expr $(expr $SECTORS \* 2) + ${BOOTIMG_EXTRA_SPACE}) + BLOCKS=$(expr $(expr $SECTORS / 2) + ${BOOTIMG_EXTRA_SPACE}) + # Ensure total sectors is an integral number of sectors per # track or mcopy will complain. Sectors are 512 bytes, and we
Signature · 1 line
1.7.5.4
PATCH 4/4 bootimg: Do not force FAT32 on all images, it violates the FAT specification
In reply to Darren HartFixes [YOCTO #1940]
do_bootimg was performing the FAT overhead calculations assuming FAT32 and then forcing the use of FAT32 with "-F 32" to mkdosfs. The FAT specification is clear on cluster count being the determining factor for FAT size (even if the fs string is set to FAT32, go figure). Syslinux follows this spec, and rightly so, resulting in a failure on core-image-minimal:
syslinux: zero FAT sectors (FAT12/16)
Drop the "-F 32" from mkdosfs to allow it to select the appropriate FAT size based on cluster count. Leave the FAT overhead calculation in FAT32. This will result in a little extra padding for really small images, but not enough extra to justify recalculating for FAT12 and FAT16.
Tested with a core-image-minimal build for atom-pc. do_bootimg completed successfully, and the resulting image was FAT16.
(From OE-Core rev: 634137704dd1a205e377a1131ef708f1c981f6b2)
Backported to edison by Darren Hart.
--- meta/classes/bootimg.bbclass | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-)
meta/classes/bootimg.bbclass +4 -1
@@ -73,6 +73,9 @@ build_boot_bin() { # Account for the filesystem overhead. This includes directory # entries in the clusters as well as the FAT itself. # Assumptions: + # FAT32 (12 or 16 may be selected by mkdosfs, but the extra + # padding will be minimal on those smaller images and not + # worth the logic here to caclulate the smaller FAT sizes) # < 16 entries per directory # 8.3 filenames only
@@ -102,7 +105,7 @@ build_boot_bin() { BLOCKS=$(expr $BLOCKS + $(expr 16 - $(expr $BLOCKS % 16))) IMG=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.hddimg - mkdosfs -F 32 -n ${BOOTIMG_VOLUME_ID} -S 512 -C ${IMG} ${BLOCKS} + mkdosfs -n ${BOOTIMG_VOLUME_ID} -S 512 -C ${IMG} ${BLOCKS} # Copy HDDDIR recursively into the image file directly mcopy -i ${IMG} -s ${HDDDIR}/* ::/
Signature · 1 line
1.7.5.4
Quoted 22 lines · Darren Hart
Backport the following patches to edison. The backport involves massaging this minimal set of patches to apply to bootimg.bbclass without pulling in the refactoring or EFI support added in master about the same time.
These patches have been tested by building atom-pc core-image-minimal with edison, a dosfsck and mount test on resulting image, and booting on the n450.
The following changes since commit
0fbd6a161576b2cafa8583adde0ffb15347c884a:poky.conf: Bumping SKD_VERSION (2012-03-14 15:49:33 -0700)are available in the git repository at:
git://git.pokylinux.org/poky-contrib dvhart/edison/bug/2138 http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dvhart/edison/bug/2138Darren Hart (4):
bootimg: Use mcopy to construct the hddimg
bootimg: Account for FAT filesystem overhead in image size
bootimg: Fix a math thinko in the block count calculation
bootimg: Do not force FAT32 on all images, it violates the FAT
specification
Quoted 3 lines
meta/classes/bootimg.bbclass | 53 ++++++++++++++++++++++++++++++++++++----- 1 files changed, 46 insertions(+), 7 deletions(-)
Thanks! I've merged these to my Edison test branch and (assuming testing goes well) will submit a consolidated pull shortly.
Cheers,
Joshua
Signature · 3 lines
Joshua '贾詡' Lock
Yocto Project "Johannes factotum"
Intel Open Source Technology Centre