wic: Fix fstype handling
cover wic: Fix fstype handling
Hi,
This patchset adds support for msdos(FAT16) fstype (#11137) and fixes number of issues in handling fstypes in wic code:
- restricted supported fstypes in wks parser code
- fixed the code that creates partitions of certain fstypes
- removed support of empty squashfs partitions
The following changes since commit 6bbf3176fe227712a478f5b4b1fdcf9417a89568:
oe-selftest: add test_image_bootpart_globbed test for wic (2017-03-30 13:27:19 +0300)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib ed/wic/wip
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/wic/wip
Ed Bartosh (9):
oe-selftest: fix incorrect fstype
wic: allow only supported fstypes
wic: don't silently skip unknown fstypes
wic: support 'msdos' fstype
wic: set FAT 16 for msdos partitions
wic: set correct system id for msdos partitions
wic: fix list of supported fstypes in help content
oe-selftest: test creation of msdos partition
wic: remove prepare_empty_partition_squashfs
meta/lib/oeqa/selftest/wic.py | 4 +- scripts/lib/wic/help.py | 2 + scripts/lib/wic/ksparser.py | 4 +- scripts/lib/wic/partition.py | 99 +++++++++++++------------------- scripts/lib/wic/plugins/imager/direct.py | 15 +---- 5 files changed, 48 insertions(+), 76 deletions(-)
--
Regards,
Ed
PATCH 1/9 oe-selftest: fix incorrect fstype
Fixed typo in wks content: squash->squashfs
--- meta/lib/oeqa/selftest/wic.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oeqa/selftest/wic.py +1 -1
@@ -729,7 +729,7 @@ part /etc --source rootfs --ondisk mmcblk0 --fstype=ext4 --exclude-path bin/ --r 'part emptyvfat --fstype vfat --size 1M\n', 'part emptyext2 --fstype ext2 --size 1M\n', 'part emptybtrfs --fstype btrfs --size 100M\n', - 'part emptysquash --fstype squash --size 1M\n']) + 'part emptysquash --fstype squashfs --size 1M\n']) wks.flush() cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) self.assertEqual(0, runCmd(cmd).status)
Signature · 1 line
2.1.4
PATCH 2/9 wic: allow only supported fstypes
In reply to Ed BartoshRestricted possible values of --fstype to the list of supported types. This should catch incorrect values when .wks file is being parsed.
Removed checks for empty fstype and mentioning of
unsupported fstype 'ontrackdm6aux3'.
--- scripts/lib/wic/ksparser.py | 4 +++- scripts/lib/wic/partition.py | 8 ++------ scripts/lib/wic/plugins/imager/direct.py | 2 -- 3 files changed, 5 insertions(+), 9 deletions(-)
scripts/lib/wic/ksparser.py +3 -1
@@ -136,7 +136,9 @@ class KickStart(): part.add_argument('--exclude-path', nargs='+') part.add_argument("--extra-space", type=sizetype) part.add_argument('--fsoptions', dest='fsopts') - part.add_argument('--fstype') + part.add_argument('--fstype', default='vfat', + choices=('ext2', 'ext3', 'ext4', 'btrfs', + 'squashfs', 'vfat', 'msdos', 'swap')) part.add_argument('--label') part.add_argument('--no-table', action='store_true') part.add_argument('--ondisk', '--ondrive', dest='disk', default='sda')
scripts/lib/wic/partition.py +2 -6
@@ -136,11 +136,11 @@ class Partition(): "specify a non-zero --size/--fixed-size for that " "partition." % self.mountpoint) - if self.fstype and self.fstype == "swap": + if self.fstype == "swap": self.prepare_swap_partition(cr_workdir, oe_builddir, native_sysroot) self.source_file = "%s/fs.%s" % (cr_workdir, self.fstype) - elif self.fstype: + else: rootfs = "%s/fs_%s.%s.%s" % (cr_workdir, self.label, self.lineno, self.fstype) if os.path.isfile(rootfs):
@@ -217,10 +217,6 @@ class Partition(): if os.path.isfile(rootfs): os.remove(rootfs) - if not self.fstype: - raise WicError("File system for partition %s not specified in " - "kickstart, use --fstype option" % self.mountpoint) - # Get rootfs size from bitbake variable if it's not set in .ks file if not self.size: # Bitbake variable ROOTFS_SIZE is calculated in
scripts/lib/wic/plugins/imager/direct.py +0 -2
@@ -487,8 +487,6 @@ class PartitionedImage(): parted_fs_type = "fat32" elif part.fstype == "msdos": parted_fs_type = "fat16" - elif part.fstype == "ontrackdm6aux3": - parted_fs_type = "ontrackdm6aux3" else: # Type for ext2/ext3/ext4/btrfs parted_fs_type = "ext2"
Signature · 1 line
2.1.4
PATCH 3/9 wic: don't silently skip unknown fstypes
In reply to Ed BartoshFixed wic code that loops through hard-coded list of known fstypes to find prepare_rootfs_<fstype> or prepare_empty_partition_<fstype> methods and silently skipping unknown fstypes.
--- scripts/lib/wic/partition.py | 32 +++++++++++++------------------- 1 file changed, 13 insertions(+), 19 deletions(-)
scripts/lib/wic/partition.py +13 -19
@@ -145,13 +145,11 @@ class Partition(): self.lineno, self.fstype) if os.path.isfile(rootfs): os.remove(rootfs) - for prefix in ("ext", "btrfs", "vfat", "squashfs"): - if self.fstype.startswith(prefix): - method = getattr(self, - "prepare_empty_partition_" + prefix) - method(rootfs, oe_builddir, native_sysroot) - self.source_file = rootfs - break + + prefix = "ext" if self.fstype.startswith("ext") else self.fstype + method = getattr(self, "prepare_empty_partition_" + prefix) + method(rootfs, oe_builddir, native_sysroot) + self.source_file = rootfs return plugins = PluginMgr.get_plugins('source')
@@ -231,19 +229,15 @@ class Partition(): '--overhead-factor will be applied') self.size = int(round(float(rsize_bb))) - for prefix in ("ext", "btrfs", "vfat", "squashfs"): - if self.fstype.startswith(prefix): - method = getattr(self, "prepare_rootfs_" + prefix) - method(rootfs, oe_builddir, rootfs_dir, native_sysroot, pseudo) - - self.source_file = rootfs + prefix = "ext" if self.fstype.startswith("ext") else self.fstype + method = getattr(self, "prepare_rootfs_" + prefix) + method(rootfs, oe_builddir, rootfs_dir, native_sysroot, pseudo) + self.source_file = rootfs - # get the rootfs size in the right units for kickstart (kB) - du_cmd = "du -Lbks %s" % rootfs - out = exec_cmd(du_cmd) - self.size = int(out.split()[0]) - - break + # get the rootfs size in the right units for kickstart (kB) + du_cmd = "du -Lbks %s" % rootfs + out = exec_cmd(du_cmd) + self.size = int(out.split()[0]) def prepare_rootfs_ext(self, rootfs, oe_builddir, rootfs_dir, native_sysroot, pseudo):
Signature · 1 line
2.1.4
PATCH 4/9 wic: support 'msdos' fstype
In reply to Ed BartoshAdded prepare_empty_partition_msdos and prepare_rootfs_msdos methods to support 'msdos' filesystem type.
Created aliases prepare_empty_partition_vfat and prepare_rootfs_vfat to continue supporting creation of vfat patitiions.
--- scripts/lib/wic/partition.py | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-)
scripts/lib/wic/partition.py +9 -5
@@ -290,10 +290,10 @@ class Partition(): (self.fstype, rootfs_size * 1024, rootfs_dir, label_str, rootfs) exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo) - def prepare_rootfs_vfat(self, rootfs, oe_builddir, rootfs_dir, - native_sysroot, pseudo): + def prepare_rootfs_msdos(self, rootfs, oe_builddir, rootfs_dir, + native_sysroot, pseudo): """ - Prepare content for a vfat rootfs partition. + Prepare content for a msdos/vfat rootfs partition. """ du_cmd = "du -bks %s" % rootfs_dir out = exec_cmd(du_cmd)
@@ -314,6 +314,8 @@ class Partition(): chmod_cmd = "chmod 644 %s" % rootfs exec_cmd(chmod_cmd) + prepare_rootfs_vfat = prepare_rootfs_msdos + def prepare_rootfs_squashfs(self, rootfs, oe_builddir, rootfs_dir, native_sysroot, pseudo): """
@@ -359,8 +361,8 @@ class Partition(): (self.fstype, self.size * 1024, label_str, rootfs) exec_native_cmd(mkfs_cmd, native_sysroot) - def prepare_empty_partition_vfat(self, rootfs, oe_builddir, - native_sysroot): + def prepare_empty_partition_msdos(self, rootfs, oe_builddir, + native_sysroot): """ Prepare an empty vfat partition. """
@@ -376,6 +378,8 @@ class Partition(): chmod_cmd = "chmod 644 %s" % rootfs exec_cmd(chmod_cmd) + prepare_empty_partition_vfat = prepare_empty_partition_msdos + def prepare_empty_partition_squashfs(self, cr_workdir, oe_builddir, native_sysroot): """
Signature · 1 line
2.1.4
PATCH 5/9 wic: set FAT 16 for msdos partitions
In reply to Ed BartoshUsed '-F 16' parameter for mkdosfs to create FAT16 partitions for 'msdos' partition type.
[YOCTO #11137]
--- scripts/lib/wic/partition.py | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
scripts/lib/wic/partition.py +12 -2
@@ -305,7 +305,12 @@ class Partition(): if self.label: label_str = "-n %s" % self.label - dosfs_cmd = "mkdosfs %s -S 512 -C %s %d" % (label_str, rootfs, rootfs_size) + size_str = "" + if self.fstype == 'msdos': + size_str = "-F 16" # FAT 16 + + dosfs_cmd = "mkdosfs %s -S 512 %s -C %s %d" % (label_str, size_str, + rootfs, rootfs_size) exec_native_cmd(dosfs_cmd, native_sysroot) mcopy_cmd = "mcopy -i %s -s %s/* ::/" % (rootfs, rootfs_dir)
@@ -372,7 +377,12 @@ class Partition(): if self.label: label_str = "-n %s" % self.label - dosfs_cmd = "mkdosfs %s -S 512 -C %s %d" % (label_str, rootfs, blocks) + size_str = "" + if self.fstype == 'msdos': + size_str = "-F 16" # FAT 16 + + dosfs_cmd = "mkdosfs %s -S 512 %s -C %s %d" % (label_str, size_str, + rootfs, blocks) exec_native_cmd(dosfs_cmd, native_sysroot) chmod_cmd = "chmod 644 %s" % rootfs
Signature · 1 line
2.1.4
PATCH 6/9 wic: set correct system id for msdos partitions
In reply to Ed BartoshExplicitly set system id 0x6(FAT16) for msdos partitions.
Removed old code that attempts to achieve the same result using 'parted ... lba off'.
--- scripts/lib/wic/plugins/imager/direct.py | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-)
scripts/lib/wic/plugins/imager/direct.py +2 -11
@@ -487,6 +487,8 @@ class PartitionedImage(): parted_fs_type = "fat32" elif part.fstype == "msdos": parted_fs_type = "fat16" + if not part.system_id: + part.system_id = '0x6' # FAT16 else: # Type for ext2/ext3/ext4/btrfs parted_fs_type = "ext2"
@@ -536,17 +538,6 @@ class PartitionedImage(): (self.path, part.num, part.system_id), self.native_sysroot) - # Parted defaults to enabling the lba flag for fat16 partitions, - # which causes compatibility issues with some firmware (and really - # isn't necessary). - if parted_fs_type == "fat16": - if self.ptable_format == 'msdos': - logger.debug("Disable 'lba' flag for partition '%s' on disk '%s'", - part.num, self.path) - exec_native_cmd("parted -s %s set %d lba off" % \ - (self.path, part.num), - self.native_sysroot) - def cleanup(self): # remove partition images for image in set(self.partimages):
Signature · 1 line
2.1.4
PATCH 7/9 wic: fix list of supported fstypes in help content
In reply to Ed BartoshAdded vfat and msdos to the list of supported fstypes in 'wic help kickstart' output.
[YOCTO #11137]
--- scripts/lib/wic/help.py | 2 ++ 1 file changed, 2 insertions(+)
scripts/lib/wic/help.py +2 -0
@@ -687,6 +687,8 @@ DESCRIPTION apply to partitions created using '--source rootfs' (see --source above). Valid values are: + vfat + msdos ext2 ext3 ext4
Signature · 1 line
2.1.4
PATCH 8/9 oe-selftest: test creation of msdos partition
In reply to Ed BartoshAdded msdos partition to the .wks file in test_fs_types wic test case.
[YOCTO #11137]
--- meta/lib/oeqa/selftest/wic.py | 1 + 1 file changed, 1 insertion(+)
meta/lib/oeqa/selftest/wic.py +1 -0
@@ -727,6 +727,7 @@ part /etc --source rootfs --ondisk mmcblk0 --fstype=ext4 --exclude-path bin/ --r 'part squash --fstype squashfs --source rootfs\n', 'part swap --fstype swap --size 1M\n', 'part emptyvfat --fstype vfat --size 1M\n', + 'part emptymsdos --fstype msdos --size 1M\n', 'part emptyext2 --fstype ext2 --size 1M\n', 'part emptybtrfs --fstype btrfs --size 100M\n', 'part emptysquash --fstype squashfs --size 1M\n'])
Signature · 1 line
2.1.4
PATCH 9/9 wic: remove prepare_empty_partition_squashfs
In reply to Ed BartoshThere is not much sense in creation of empty squashfs partition. It's also not possible to create empty squashfs partition of specified size.
Even more, prepare_empty_partition_squashfs method is absolutely broken. It raises exception when called and even its signature differs from the rest of of similar methods. It means that nobody uses it and it's safe to remove it.
Removed prepare_empty_partition_squashfs method and testing of empty squashfs partition.
--- meta/lib/oeqa/selftest/wic.py | 3 +-- scripts/lib/wic/partition.py | 33 ++++----------------------------- 2 files changed, 5 insertions(+), 31 deletions(-)
meta/lib/oeqa/selftest/wic.py +1 -2
@@ -729,8 +729,7 @@ part /etc --source rootfs --ondisk mmcblk0 --fstype=ext4 --exclude-path bin/ --r 'part emptyvfat --fstype vfat --size 1M\n', 'part emptymsdos --fstype msdos --size 1M\n', 'part emptyext2 --fstype ext2 --size 1M\n', - 'part emptybtrfs --fstype btrfs --size 100M\n', - 'part emptysquash --fstype squashfs --size 1M\n']) + 'part emptybtrfs --fstype btrfs --size 100M\n']) wks.flush() cmd = "wic create %s -e %s -o %s" % (wks.name, img, self.resultdir) self.assertEqual(0, runCmd(cmd).status)
scripts/lib/wic/partition.py +4 -29
@@ -141,6 +141,10 @@ class Partition(): native_sysroot) self.source_file = "%s/fs.%s" % (cr_workdir, self.fstype) else: + if self.fstype == 'squashfs': + raise WicError("It's not possible to create empty squashfs " + "partition '%s'" % (self.mountpoint)) + rootfs = "%s/fs_%s.%s.%s" % (cr_workdir, self.label, self.lineno, self.fstype) if os.path.isfile(rootfs):
@@ -390,35 +394,6 @@ class Partition(): prepare_empty_partition_vfat = prepare_empty_partition_msdos - def prepare_empty_partition_squashfs(self, cr_workdir, oe_builddir, - native_sysroot): - """ - Prepare an empty squashfs partition. - """ - logger.warning("Creating of an empty squashfs %s partition was attempted. " - "Proceeding as requested.", self.mountpoint) - - path = "%s/fs_%s.%s" % (cr_workdir, self.label, self.fstype) - if os.path.isfile(path): - os.remove(path) - - # it is not possible to create a squashfs without source data, - # thus prepare an empty temp dir that is used as source - tmpdir = tempfile.mkdtemp() - - squashfs_cmd = "mksquashfs %s %s -noappend" % \ - (tmpdir, path) - exec_native_cmd(squashfs_cmd, native_sysroot) - - os.rmdir(tmpdir) - - # get the rootfs size in the right units for kickstart (kB) - du_cmd = "du -Lbks %s" % path - out = exec_cmd(du_cmd) - fs_size = out.split()[0] - - self.size = int(fs_size) - def prepare_swap_partition(self, cr_workdir, oe_builddir, native_sysroot): """ Prepare a swap partition.
Signature · 1 line
2.1.4
✗ patchtest: failure for wic: Fix fstype handling
In reply to Ed Bartosh== Series Details ==
Series: wic: Fix fstype handling
Revision: 1
URL : https://patchwork.openembedded.org/series/6091/
State : failure
== Summary ==
Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures:
* Issue Series does not apply on top of target branch [test_series_merge_on_head]
Suggested fix Rebase your series on top of targeted branch
Targeted branch master (currently at 6d6d0737a2)
If you believe any of these test results are incorrect, please reply to the mailing list ([email protected]) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...).
---
Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe