dosfstools upgrade + image gen. fixes
cover dosfstools upgrade + image gen. fixes
A new approach to the dosfstools upgrade: Fix the sanity check in mtools and remove workarounds we have in image generation scripts.
I ended up at this after I realised that:
* dosfstools < 4.0 uses 32 sectors-per-track
* dosfstools 4.0 uses 63 sectors-per-track
* dosfstools git master uses variable sectors-per-track
Creating a filesystem with a size that's divisible by sectors-per-track with multiple different dosfstools (we have two versions) was going to be very tricky.
On the other hand, making the sanity check in mtools non-fatal seems to work just fine and let's us simplify the scripts.
- Jussi
The following changes since commit 5a9745dd631eb697c30cd49e1dd065f4f57e7393:
distro_check.py: Don't mix tabs and spaces (2016-06-02 08:36:29 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib jku/dosfstools
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=jku/dosfstools
Jussi Kukkonen (3):
mtools: Patch out a useless sanity check image-live, image-vm, wic: Remove fs size workaround
dosfstools: Upgrade 3.0.28 -> 4.0
meta/classes/image-live.bbclass | 6 ---
meta/classes/image-vm.bbclass | 6 ---
.../0001-mkfs.fat-fix-incorrect-int-type.patch | 46 ----------------------
.../dosfstools/dosfstools/largefile.patch | 19 ---------
.../{dosfstools_3.0.28.bb => dosfstools_4.0.bb} | 18 +++------
...en-if-fs-size-is-not-divisible-by-sectors.patch | 34 ++++++++++++++++
meta/recipes-devtools/mtools/mtools_3.9.9.bb | 4 +-
meta/recipes-devtools/mtools/mtools_4.0.18.bb | 4 +-
scripts/lib/wic/partition.py | 8 ----
scripts/lib/wic/plugins/source/bootimg-efi.py | 6 ---
scripts/lib/wic/plugins/source/bootimg-pcbios.py | 6 ---
.../lib/wic/plugins/source/isoimage-isohybrid.py | 6 ---
12 files changed, 46 insertions(+), 117 deletions(-)
delete mode 100644 meta/recipes-devtools/dosfstools/dosfstools/0001-mkfs.fat-fix-incorrect-int-type.patch
delete mode 100644 meta/recipes-devtools/dosfstools/dosfstools/largefile.patch
rename meta/recipes-devtools/dosfstools/{dosfstools_3.0.28.bb => dosfstools_4.0.bb} (60%)
create mode 100644 meta/recipes-devtools/mtools/mtools/0001-Continue-even-if-fs-size-is-not-divisible-by-sectors.patchSignature · 1 line
2.1.4
PATCH 1/3 mtools: Patch out a useless sanity check
The sanity check is ensuring that filesystem size is divisible by sectors-per-track. That seems to be just an optimization for ancient media, and quite tricky to accomplish as different dosfstools versions produce filesystems with different sectors-per-track.
Continue processing even if filesystem size is not divisible by sectors-per-track.
--- ...en-if-fs-size-is-not-divisible-by-sectors.patch | 34 ++++++++++++++++++++++ meta/recipes-devtools/mtools/mtools_3.9.9.bb | 4 ++- meta/recipes-devtools/mtools/mtools_4.0.18.bb | 4 ++- 3 files changed, 40 insertions(+), 2 deletions(-) create mode 100644 meta/recipes-devtools/mtools/mtools/0001-Continue-even-if-fs-size-is-not-divisible-by-sectors.patch
meta/recipes-devtools/mtools/mtools/0001-Continue-even-if-fs-size-is-not-divisible-by-sectors.patch new file +34 -0
@@ -0,0 +1,34 @@ +From 5cc8d1f96ab6d3459e13631d8356f87381562352 Mon Sep 17 00:00:00 2001 +From: Jussi Kukkonen <[email protected]> +Date: Thu, 2 Jun 2016 09:49:44 +0300 +Subject: [PATCH] Continue even if fs size is not divisible by + sectors-per-track + +Filesystem with a size that is not divisible by sectors-per-track is +not broken or invalid: it might be unoptimized on media where +heads and cylinders actually matter but that should be it. + +Preserve the warning but make mtools continue even if fs size is not +divisible by sectors-per-track. + +Upstream-Status: Submitted [[email protected]] +Signed-off-by: Jussi Kukkonen <[email protected]> +--- + init.c | 1 - + 1 file changed, 1 deletion(-) + +diff --git a/init.c b/init.c +index c9152e0..4e16a01 100644 +--- a/init.c ++++ b/init.c +@@ -338,7 +338,6 @@ Stream_t *fs_init(char drive, int mode, int *isRop) + fprintf(stderr, + "Add mtools_skip_check=1 to your .mtoolsrc file " + "to skip this test\n"); +- exit(1); + } + + /* full cylinder buffering */ +-- +2.1.4 +
meta/recipes-devtools/mtools/mtools_3.9.9.bb +3 -1
@@ -33,7 +33,9 @@ SRC_URI = "http://downloads.yoctoproject.org/mirror/sources/mtools-${PV}.tar.gz file://mtools-makeinfo.patch \ file://mtools.patch \ file://no-x11.patch \ - file://fix-broken-lz.patch" + file://fix-broken-lz.patch \ + file://0001-Continue-even-if-fs-size-is-not-divisible-by-sectors.patch \ + " inherit autotools texinfo
meta/recipes-devtools/mtools/mtools_4.0.18.bb +3 -1
@@ -29,7 +29,9 @@ SRC_URI[sha256sum] = "59e9cf80885399c4f229e5d87e49c0c2bfeec044e1386d59fcd0b0aead SRC_URI = "${GNU_MIRROR}/mtools/mtools-${PV}.tar.bz2 \ file://mtools-makeinfo.patch \ - file://no-x11.gplv3.patch" + file://no-x11.gplv3.patch \ + file://0001-Continue-even-if-fs-size-is-not-divisible-by-sectors.patch \ + " inherit autotools texinfo
Signature · 1 line
2.1.4
PATCH 2/3 image-live, image-vm, wic: Remove fs size workaround
Since mtools has been patched to live with filesystems with sizes not divisible by sectors-per-track, we no longer need to try to set the size based on our guess of the sectors-per-track dosfstools is going to use.
--- meta/classes/image-live.bbclass | 6 ------ meta/classes/image-vm.bbclass | 6 ------ scripts/lib/wic/partition.py | 8 -------- scripts/lib/wic/plugins/source/bootimg-efi.py | 6 ------ scripts/lib/wic/plugins/source/bootimg-pcbios.py | 6 ------ scripts/lib/wic/plugins/source/isoimage-isohybrid.py | 6 ------ 6 files changed, 38 deletions(-)
meta/classes/image-live.bbclass +0 -6
@@ -202,12 +202,6 @@ build_fat_img() { # Determine the final size in blocks accounting for some padding BLOCKS=$(expr $(expr $SECTORS / 2) + ${BOOTIMG_EXTRA_SPACE}) - # Ensure total sectors is an integral number of sectors per - # track or mcopy will complain. Sectors are 512 bytes, and we - # generate images with 32 sectors per track. This calculation is - # done in blocks, thus the mod by 16 instead of 32. - BLOCKS=$(expr $BLOCKS + $(expr 16 - $(expr $BLOCKS % 16))) - # mkdosfs will sometimes use FAT16 when it is not appropriate, # resulting in a boot failure from SYSLINUX. Use FAT32 for # images larger than 512MB, otherwise let mkdosfs decide.
meta/classes/image-vm.bbclass +0 -6
@@ -66,12 +66,6 @@ build_boot_dd() { BLOCKS=`du -bks $HDDDIR | cut -f 1` BLOCKS=`expr $BLOCKS + ${BOOTDD_EXTRA_SPACE}` - # Ensure total sectors is an integral number of sectors per - # track or mcopy will complain. Sectors are 512 bytes, and we - # generate images with 32 sectors per track. This calculation is - # done in blocks, thus the mod by 16 instead of 32. - BLOCKS=$(expr $BLOCKS + $(expr 16 - $(expr $BLOCKS % 16))) - # Remove it since mkdosfs would fail when it exists rm -f $HDDIMG mkdosfs -n ${BOOTDD_VOLUME_ID} -S 512 -C $HDDIMG $BLOCKS
scripts/lib/wic/partition.py +0 -8
@@ -281,14 +281,6 @@ class Partition(): msger.debug("Added %d extra blocks to %s to get to %d total blocks" % \ (extra_blocks, self.mountpoint, blocks)) - # Ensure total sectors is an integral number of sectors per - # track or mcopy will complain. Sectors are 512 bytes, and we - # generate images with 32 sectors per track. This calculation - # is done in blocks, thus the mod by 16 instead of 32. Apply - # sector count fix only when needed. - if blocks % 16 != 0: - blocks += (16 - (blocks % 16)) - label_str = "-n boot" if self.label: label_str = "-n %s" % self.label
scripts/lib/wic/plugins/source/bootimg-efi.py +0 -6
@@ -216,12 +216,6 @@ class BootimgEFIPlugin(SourcePlugin): msger.debug("Added %d extra blocks to %s to get to %d total blocks" % \ (extra_blocks, part.mountpoint, blocks)) - # Ensure total sectors is an integral number of sectors per - # track or mcopy will complain. Sectors are 512 bytes, and we - # generate images with 32 sectors per track. This calculation is - # done in blocks, thus the mod by 16 instead of 32. - blocks += (16 - (blocks % 16)) - # dosfs image, created by mkdosfs bootimg = "%s/boot.img" % cr_workdir
scripts/lib/wic/plugins/source/bootimg-pcbios.py +0 -6
@@ -179,12 +179,6 @@ class BootimgPcbiosPlugin(SourcePlugin): msger.debug("Added %d extra blocks to %s to get to %d total blocks" % \ (extra_blocks, part.mountpoint, blocks)) - # Ensure total sectors is an integral number of sectors per - # track or mcopy will complain. Sectors are 512 bytes, and we - # generate images with 32 sectors per track. This calculation is - # done in blocks, thus the mod by 16 instead of 32. - blocks += (16 - (blocks % 16)) - # dosfs image, created by mkdosfs bootimg = "%s/boot.img" % cr_workdir
scripts/lib/wic/plugins/source/isoimage-isohybrid.py +0 -6
@@ -441,12 +441,6 @@ class IsoImagePlugin(SourcePlugin): % (part.mountpoint, blocks) msger.debug(msg) - # Ensure total sectors is an integral number of sectors per - # track or mcopy will complain. Sectors are 512 bytes, and we - # generate images with 32 sectors per track. This calculation is - # done in blocks, thus the mod by 16 instead of 32. - blocks += (16 - (blocks % 16)) - # dosfs image for EFI boot bootimg = "%s/efi.img" % isodir
Signature · 1 line
2.1.4
PATCH 3/3 dosfstools: Upgrade 3.0.28 -> 4.0
In reply to Jussi Kukkonen* Build system is now autotools
* Patches are no longer required
* Enable compatibility symlinks
---
.../0001-mkfs.fat-fix-incorrect-int-type.patch | 46 ----------------------
.../dosfstools/dosfstools/largefile.patch | 19 ---------
.../{dosfstools_3.0.28.bb => dosfstools_4.0.bb} | 18 +++------
3 files changed, 6 insertions(+), 77 deletions(-)
delete mode 100644 meta/recipes-devtools/dosfstools/dosfstools/0001-mkfs.fat-fix-incorrect-int-type.patch
delete mode 100644 meta/recipes-devtools/dosfstools/dosfstools/largefile.patch
rename meta/recipes-devtools/dosfstools/{dosfstools_3.0.28.bb => dosfstools_4.0.bb} (60%)meta/recipes-devtools/dosfstools/dosfstools/0001-mkfs.fat-fix-incorrect-int-type.patch deleted +0 -46
@@ -1,46 +0,0 @@ -From 3b95786af13e28157d889bd90a384ee255f2d91d Mon Sep 17 00:00:00 2001 -From: "Yann E. MORIN" <[email protected]> -Date: Sun, 16 Aug 2015 15:55:43 +0200 -Subject: [PATCH] mkfs.fat: fix incorrect int type - -u_int32_t is not a stanard type, while uint32_t is. This fixes builds -with the musl C library, which only defines so-called "clean" headers; -build failures are like (back-quotes and elision manually added for -readability): - - http://autobuild.buildroot.org/results/a09/a0923d7f6d4dbae02eba4c5024bbdae3a52aa85a/build-end.log - - /home/peko/autobuild/instance-1/output/host/usr/bin/x86_64-linux-gcc -D_LARGEFILE_SOURCE \ - -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -D_GNU_SOURCE -D_LARGEFILE_SOURCE \ - -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -c -o mkfs.fat.o src/mkfs.fat.c - src/mkfs.fat.c: In function 'main': - src/mkfs.fat.c:1415:18: error: 'u_int32_t' undeclared (first use in this function) - volume_id = (u_int32_t) ((create_timeval.tv_sec << 20) | create_timeval.tv_usec); [...] - ^ - src/mkfs.fat.c:1415:18: note: each undeclared identifier is reported only once for each - function it appears in - -Signed-off-by: "Yann E. MORIN" <[email protected]> -Signed-off-by: Andreas Bombe <[email protected]> ---- -Upstream-Status: Backport - - src/mkfs.fat.c | 2 +- - 1 file changed, 1 insertion(+), 1 deletion(-) - -diff --git a/src/mkfs.fat.c b/src/mkfs.fat.c -index b38d116..dddbe24 100644 ---- a/src/mkfs.fat.c -+++ b/src/mkfs.fat.c -@@ -1412,7 +1412,7 @@ int main(int argc, char **argv) - - gettimeofday(&create_timeval, NULL); - create_time = create_timeval.tv_sec; -- volume_id = (u_int32_t) ((create_timeval.tv_sec << 20) | create_timeval.tv_usec); /* Default volume ID = creation time, fudged for more uniqueness */ -+ volume_id = (uint32_t) ((create_timeval.tv_sec << 20) | create_timeval.tv_usec); /* Default volume ID = creation time, fudged for more uniqueness */ - check_atari(); - - printf("mkfs.fat " VERSION " (" VERSION_DATE ")\n"); --- -2.7.0 -
meta/recipes-devtools/dosfstools/dosfstools/largefile.patch deleted +0 -19
@@ -1,19 +0,0 @@ -defer setting largefile flags to build makery -in a cross build we can not do runtime config tests - -Signed-off-by: Khem Raj <[email protected]> -Upstream-Status: Inappropriate[Cross compile Specific] -Index: dosfstools-3.0.28/Makefile -=================================================================== ---- dosfstools-3.0.28.orig/Makefile -+++ dosfstools-3.0.28/Makefile -@@ -27,8 +27,7 @@ SBINDIR = $(PREFIX)/sbin - DOCDIR = $(PREFIX)/share/doc - MANDIR = $(PREFIX)/share/man - --#OPTFLAGS = -O2 -fomit-frame-pointer -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 --OPTFLAGS = -O2 -fomit-frame-pointer -D_GNU_SOURCE $(shell getconf LFS_CFLAGS) -+OPTFLAGS = -O2 -fomit-frame-pointer - #WARNFLAGS = -Wall -pedantic -std=c99 - WARNFLAGS = -Wall -Wextra -Wno-sign-compare -Wno-missing-field-initializers -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings - DEBUGFLAGS = -g
meta/recipes-devtools/dosfstools/dosfstools_4.0.bb renamed from meta/recipes-devtools/dosfstools/dosfstools_3.0.28.bb +6 -12
@@ -9,23 +9,17 @@ SECTION = "base" LICENSE = "GPLv3" LIC_FILES_CHKSUM = "file://COPYING;md5=d32239bcb673463ab874e80d47fae504" -SRC_URI = "https://github.com/dosfstools/dosfstools/releases/download/v3.0.28/dosfstools-3.0.28.tar.xz \ - file://largefile.patch \ - file://0001-mkfs.fat-fix-incorrect-int-type.patch \ +SRC_URI = "https://github.com/dosfstools/dosfstools/releases/download/v${PV}/${BP}.tar.xz \ " -SRC_URI[md5sum] = "6a047a6c65186b9ebb1853709adb36db" -SRC_URI[sha256sum] = "ee95913044ecf2719b63ea11212917649709a6e53209a72d622135aaa8517ee2" +SRC_URI[md5sum] = "20f8388b99702f276c973d228c7cff45" +SRC_URI[sha256sum] = "9037738953559d1efe04fc5408b6846216cc0138f7f9d32de80b6ec3c35e7daf" UPSTREAM_CHECK_URI = "https://github.com/dosfstools/dosfstools/releases" -CFLAGS += "-D_GNU_SOURCE ${@bb.utils.contains('DISTRO_FEATURES', 'largefile', '-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', '', d)}" +inherit autotools pkgconfig -FILES_${PN} = "${base_sbindir}" -FILES_${PN}-doc = "${mandir} ${docdir}" +EXTRA_OECONF = "--without-udev --enable-compat-symlinks" -do_install () { - oe_runmake "PREFIX=${D}${prefix}" "SBINDIR=${D}${base_sbindir}" \ - "MANDIR=${D}${mandir}" "DOCDIR=${D}${docdir}" install -} +CFLAGS += "-D_GNU_SOURCE ${@bb.utils.contains('DISTRO_FEATURES', 'largefile', '-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64', '', d)}" BBCLASSEXTEND = "native"
Signature · 1 line
2.1.4