Creating a sparse file in ROOTFS_F2FS_CMD
Subsystems (2)
cover Creating a sparse file in ROOTFS_F2FS_CMD
I'm not sure if creating a sparse file using dd command from host OS is acceptable. Maybe it's needed to find a better solution.
Grzegorz Blach (2):
package/f2fs-tools: add host package
fs/f2fs: add support for creating a f2fs image
DEVELOPERS | 1 + fs/Config.in | 1 + fs/f2fs/Config.in | 49 +++++++++++++++++++++++++++++++ fs/f2fs/f2fs.mk | 46 +++++++++++++++++++++++++++++ package/Config.in.host | 1 + package/f2fs-tools/Config.in.host | 7 +++++ package/f2fs-tools/f2fs-tools.mk | 2 ++ 7 files changed, 107 insertions(+) create mode 100644 fs/f2fs/Config.in create mode 100644 fs/f2fs/f2fs.mk create mode 100644 package/f2fs-tools/Config.in.host
Signature · 1 line
2.17.2
PATCH 1/2 package/f2fs-tools: add host package
This package is useful to create f2fs filesystem during build process.
--- DEVELOPERS | 1 + package/Config.in.host | 1 + package/f2fs-tools/Config.in.host | 7 +++++++ package/f2fs-tools/f2fs-tools.mk | 2 ++ 4 files changed, 11 insertions(+) create mode 100644 package/f2fs-tools/Config.in.host
DEVELOPERS +1 -0
@@ -891,6 +891,7 @@ F: package/sofia-sip/ N: Grzegorz Blach <[email protected]> F: package/bluez5_utils-headers/ +F: package/f2fs-tools/ F: package/pigpio/ F: package/python-falcon/ F: package/python-mimeparse/
package/Config.in.host +1 -0
@@ -16,6 +16,7 @@ menu "Host utilities" source "package/dtc/Config.in.host" source "package/e2fsprogs/Config.in.host" source "package/e2tools/Config.in.host" + source "package/f2fs-tools/Config.in.host" source "package/faketime/Config.in.host" source "package/fwup/Config.in.host" source "package/genext2fs/Config.in.host"
package/f2fs-tools/Config.in.host new file +7 -0
@@ -0,0 +1,7 @@ +config BR2_PACKAGE_HOST_F2FS_TOOLS + bool "host f2fs-tools" + select BR2_PACKAGE_HOST_UTIL_LINUX + help + Tools for Flash-Friendly File System (F2FS) + + https://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-tools.git
package/f2fs-tools/f2fs-tools.mk +2 -0
@@ -9,6 +9,7 @@ F2FS_TOOLS_SITE = http://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-to F2FS_TOOLS_SITE_METHOD = git F2FS_TOOLS_CONF_ENV = ac_cv_file__git=no F2FS_TOOLS_DEPENDENCIES = host-pkgconf util-linux +HOST_F2FS_TOOLS_DEPENDENCIES = host-pkgconf host-util-linux # GIT version, shipped without configure F2FS_TOOLS_AUTORECONF = YES F2FS_TOOLS_INSTALL_STAGING = YES
@@ -30,3 +31,4 @@ F2FS_TOOLS_CONF_OPTS += --without-blkid endif $(eval $(autotools-package)) +$(eval $(host-autotools-package))
Signature · 1 line
2.17.2
PATCH 1/2 package/f2fs-tools: add host package
Grzegorz, All,
On 2018-10-26 22:00 +0200, Grzegorz Blach spake thusly:
Quoted 1 line
This package is useful to create f2fs filesystem during build process.
Indeed.
Quoted 2 lines
Signed-off-by Grzegorz Blach <[email protected]>---
[--SNIP--]
Quoted 13 lines
package/f2fs-tools/f2fs-tools.mk +1
-0@@ -9,6 +9,7 @@ F2FS_TOOLS_SITE = http://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs-to F2FS_TOOLS_SITE_METHOD = git F2FS_TOOLS_CONF_ENV = ac_cv_file__git=no F2FS_TOOLS_DEPENDENCIES = host-pkgconf util-linux +HOST_F2FS_TOOLS_DEPENDENCIES = host-pkgconf host-util-linux # GIT version, shipped without configure F2FS_TOOLS_AUTORECONF = YES F2FS_TOOLS_INSTALL_STAGING = YES
package/f2fs-tools/f2fs-tools.mk F2FS_TOOLS_CONF_OPTS += --without-blkid · line 31
Quoted 1 line
package/f2fs-tools/f2fs-tools.mk +0
-0@@ -30,3 +31,4 @@ F2FS_TOOLS_CONF_OPTS += --without-blkid endif
I don't think that libblkid is needed for the host variant, so we should probably explicitly disable it. On the other hand, libblkid is always built by host-util-linux.
Whatever yuou choose, make that explicit.
Also, we do have host-libselinux, so wyou may want to also enable that unconditionally.
HOST_F2FS_TOOLS_DEPENDENCIES = \
host-libselinux \
host-pkgconf \
host-util-linux
HOST_F2FS_TOOLS_CONF_OPTS = --without-blkid --with-selinuxRegards,
Yann E. MORIN.
Quoted 9 lines
package/f2fs-tools/f2fs-tools.mk +1
-0$(eval $(autotools-package)) +$(eval $(host-autotools-package))Signature · 6 lines
2.17.2 _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 1/2 package/f2fs-tools: add host package
Hello,
Quoted 3 lines · Yann E. MORIN
I don't think that libblkid is needed for the host variant, so we should probably explicitly disable it. On the other hand, libblkid is always built by host-util-linux.
Agreed, I've added --without-blkid.
Quoted 2 lines
Also, we do have host-libselinux, so wyou may want to also enable that unconditionally.
I don't really agree here. We try to build only a minimal set of dependencies, so we shouldn't build against host-libselinux by default. Yes, solving that properly requires introducing Config.in options for host packages :-)
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/2 package/f2fs-tools: add host package
In reply to Grzegorz BlachHello,
Quoted 3 lines · Grzegorz Blach
This package is useful to create f2fs filesystem during build process.
Signed-off-by Grzegorz Blach <[email protected]>
Thanks, I have applied, after adding --without-selinux --without-blkid to HOST_F2FS_TOOLS_CONF_OPTS. --without-blkid was suggested by Yann. However, contrary to Yann, I don't think we should unconditionally enable SELinux support, so I've added a --without-selinux.
Thanks,
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 2/2 fs/f2fs: add support for creating a f2fs image
In reply to Grzegorz BlachThis patch makes possible to create rootfs image using f2fs filesystem.
--- fs/Config.in | 1 + fs/f2fs/Config.in | 49 +++++++++++++++++++++++++++++++++++++++++++++++ fs/f2fs/f2fs.mk | 46 ++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 96 insertions(+) create mode 100644 fs/f2fs/Config.in create mode 100644 fs/f2fs/f2fs.mk
fs/Config.in +1 -0
@@ -6,6 +6,7 @@ source "fs/cloop/Config.in" source "fs/cpio/Config.in" source "fs/cramfs/Config.in" source "fs/ext2/Config.in" +source "fs/f2fs/Config.in" source "fs/initramfs/Config.in" source "fs/iso9660/Config.in" source "fs/jffs2/Config.in"
fs/f2fs/Config.in new file +49 -0
49 lines not on this page · raw message
fs/f2fs/f2fs.mk new file +46 -0
46 lines not on this page · raw message
Signature · 1 line
2.17.2
PATCH 2/2 fs/f2fs: add support for creating a f2fs image
Grzegorz, All,
On 2018-10-26 22:00 +0200, Grzegorz Blach spake thusly:
Quoted 4 lines
This patch makes possible to create rootfs image using f2fs filesystem.
Signed-off-by Grzegorz Blach <[email protected]>---
[--SNIP--]
Quoted 22 lines
fs/f2fs/Config.in new file +16
-0@@ -0,0 +1,49 @@ +config BR2_TARGET_ROOTFS_F2FS + bool "f2fs root filesystem" + select BR2_PACKAGE_HOST_F2FS_TOOLS + help + Build a f2fs root filesystem. If you enable this option, you + probably want to enable the f2fs-tools package too. + +if BR2_TARGET_ROOTFS_F2FS + +config BR2_TARGET_ROOTFS_F2FS_LABEL + string "filesystem label" + +config BR2_TARGET_ROOTFS_F2FS_SIZE + string "filesystem size" + default "100M" + help
fs/f2fs/Config.in line 26
Quoted 10 lines
fs/f2fs/Config.in new file +10
-0+ The size of the filesystem image in bytes. + Suffix with K, M, G or T for power-of-two kilo-, mega-, giga- + or terabytes. + +config BR2_TARGET_ROOTFS_F2FS_COLD_FILES + string "extension list for cold files" + help + Specify a file extension list in order f2fs to treat them + as cold files. The default list includes most of multimedia + file extensions such as jpg, gif, mpeg, mkv, and so on.
I would prefer that we get a first patch that adds the bare minimal needed to assemble an f2fs image, with further patches adding more features one by one, as it is easier to review.
This "cold files" feature should be added in its own patch.
fs/f2fs/Config.in line 35
Quoted 8 lines
fs/f2fs/Config.in new file +8
-0+config BR2_TARGET_ROOTFS_F2FS_OVERPROVISION + int "size for overprovision area (0 for auto calculation)" + default 0 + help + Specify the percentage over the volume size for overprovision + area. This area is hidden to users, and utilized by F2FS + cleaner. If not specified, the best number will be assigned + automatically accoring to the partition size.
May I suggest an alternative wording?
int "size for overprovision area"
default 0
help
The percentage over the volume size for overprovision area. This area is hidden to users, and utilized by F2FS cleaner.
Leave at 0 for autocalculation.
But this feature should be added in its own patch.
fs/f2fs/Config.in line 41
Quoted 5 lines
fs/f2fs/Config.in new file +5
-0+config BR2_TARGET_ROOTFS_F2FS_FEATURES + string "filesystem features" + help + A feature list in order f2fs filesystem will supports. + e.g "encrypt" and so on.
Ditto, a separate patch. Also, this option being a plain string, I'd prefer it goes last in the list of options, below the discard one.
fs/f2fs/Config.in line 47
Quoted 5 lines
fs/f2fs/Config.in new file +5
-0+config BR2_TARGET_ROOTFS_F2FS_DISCARD + bool "discard policy" + default y + help + Enable or disable discard policy.
Ditto, a separate patch for that one, please.
Quoted 32 lines
fs/f2fs/Config.in new file +1
-0+endif # BR2_TARGET_ROOTFS_F2FSfs/f2fs/f2fs.mk new file +25
-0@@ -0,0 +1,46 @@ +################################################################################ +# +# Build the f2fs root filesystem image +# +################################################################################ + +F2FS_SIZE = $(call qstrip,$(BR2_TARGET_ROOTFS_F2FS_SIZE)) +ifeq ($(BR2_TARGET_ROOTFS_F2FS)-$(F2FS_SIZE),y-) +$(error BR2_TARGET_ROOTFS_F2FS_SIZE cannot be empty) +endif + +F2FS_COLD_FILES = $(call qstrip,$(BR2_TARGET_ROOTFS_F2FS_COLD_FILES)) +F2FS_FEATURES = $(call qstrip,$(BR2_TARGET_ROOTFS_F2FS_FEATURES)) +# qstrip results in stripping consecutive spaces into a single one. So the +# variable is not qstrip-ed to preserve the integrity of the string value. +F2FS_LABEL := $(subst ",,$(BR2_TARGET_ROOTFS_F2FS_LABEL)) +# ") + +ifneq ($(BR2_TARGET_ROOTFS_F2FS_OVERPROVISION),0) +F2FS_OVERPROVISION=$(BR2_TARGET_ROOTFS_F2FS_OVERPROVISION) +endif + +ifeq ($(BR2_TARGET_ROOTFS_F2FS_DISCARD),y) +F2FS_DISCARD=1 +else
fs/f2fs/f2fs.mk line 35
Quoted 10 lines
fs/f2fs/f2fs.mk new file +10
-0+F2FS_DISCARD=0 +endif + +F2FS_OPTS = \ + -f \ + -l "$(F2FS_LABEL)" \ + $(if $(F2FS_COLD_FILES),-e "$(F2FS_COLD_FILES)") \ + $(if $(F2FS_OVERPROVISION),-o $(F2FS_OVERPROVISION)) \ + $(if $(F2FS_FEATURES),-O "$(F2FS_FEATURES)") \ + -t $(F2FS_DISCARD)
Conditional values should go last:
F2FS_OPTS = \
-f \
-l "$(F2FS_LABEL)" \
-t $(F2FS_DISCARD) \
$(if $(F2FS_COLD_FILES),-e "$(F2FS_COLD_FILES)") \
[...]fs/f2fs/f2fs.mk line 41
Quoted 5 lines
fs/f2fs/f2fs.mk new file +5
-0+ROOTFS_F2FS_DEPENDENCIES = host-f2fs-tools + +define ROOTFS_F2FS_CMD + $(RM) -f $@ + dd if=/dev/zero of=$@ bs=1 count=0 seek=$(F2FS_SIZE)
You mentionned in your cover letter that you were not sure using dd was the best solution. It is acceptable, but usng truncate might be a better option, and you can alsoirectly pass it the size value:
truncate -s '$(F2FS_SIZE)' $@
Otherwise, when using dd to achieve the same, I invert bs and seek:
(i.e. it is 1 bloc of N, not N blocs of 1).
dd if=/dev/zero of=$@ bs=$(F2FS_SIZE) count=0 seek=1
So, dd is acceptable, but I'd favour truncate.
Regards,
Yann E. MORIN.
Quoted 12 lines
fs/f2fs/f2fs.mk new file +5
-0+ $(HOST_DIR)/sbin/mkfs.f2fs $(F2FS_OPTS) $@ + $(HOST_DIR)/sbin/sload.f2fs -f $(TARGET_DIR) $@ +endef + +$(eval $(rootfs))Signature · 6 lines
2.17.2 _______________________________________________ buildroot mailing list buildroot at busybox.net http://lists.busybox.net/mailman/listinfo/buildroot
Signature · 6 lines
.-----------------.--------------------.------------------.--------------------. | Yann E. MORIN | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: | | +33 662 376 056 | Software Designer | \ / CAMPAIGN | ___ | | +33 223 225 172 `------------.-------: X AGAINST | \e/ There is no | | http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL | v conspiracy. | '------------------------------^-------^------------------^--------------------'
PATCH 2/2 fs/f2fs: add support for creating a f2fs image
In reply to Grzegorz BlachHello,
Quoted 3 lines · Grzegorz Blach
This patch makes possible to create rootfs image using f2fs filesystem.
Signed-off-by Grzegorz Blach <[email protected]>
I have applied, after doing a number of changes, mainly splitting into separate patches, as suggested by Yann E. Morin.
However, now that you have added F2FS support, it would be good if you could add a test in our test suite for this filesystem. Look at support/testing/tests/fs/ for the other filesystem tests that we have. Could you add something like this ?
fs/f2fs/Config.in line 22
Quoted 2 lines
fs/f2fs/Config.in new file +2
-0+config BR2_TARGET_ROOTFS_F2FS_COLD_FILES + string "extension list for cold files"
Any reason for having added this option for cold files, but not the symmetric option for hot files ?
fs/f2fs/Config.in line 26
Quoted 4 lines
fs/f2fs/Config.in new file +4
-0+ help + Specify a file extension list in order f2fs to treat them + as cold files. The default list includes most of multimedia + file extensions such as jpg, gif, mpeg, mkv, and so on.
I've seen this after committing, but the help text of mkfs.f2fs seems to imply that the list of extensions should be comma separated. If that's the case, it should be mentioned in the help text, because most Buildroot options use space-separated lists, not comma-separated ones.
fs/f2fs/Config.in line 29
Quoted 2 lines
fs/f2fs/Config.in new file +2
-0+config BR2_TARGET_ROOTFS_F2FS_OVERPROVISION + int "size for overprovision area (0 for auto calculation)"
Actually 0 is not auto-calculation. 0 will not pass any -O option, and therefore mkfs.f2fs will use its default of 5%.
Also, this option doesn't give the size, but the ratio.
I've fixed both aspects when committing.
fs/f2fs/f2fs.mk line 41
Quoted 1 line
fs/f2fs/f2fs.mk new file +1
-0+ dd if=/dev/zero of=$@ bs=1 count=0 seek=$(F2FS_SIZE)
I've replaced by a truncate, as suggested by Yann.
Thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 2/2 fs/f2fs: add support for creating a f2fs image
Quoted 7 lines · Thomas Petazzoni
Hello,
Quoted 3 lines · Grzegorz Blach
This patch makes possible to create rootfs image using f2fs filesystem.
Signed-off-by Grzegorz Blach <[email protected]>
Quoted 8 lines
I have applied, after doing a number of changes, mainly splitting into separate patches, as suggested by Yann E. Morin.
However, now that you have added F2FS support, it would be good if you could add a test in our test suite for this filesystem. Look at support/testing/tests/fs/ for the other filesystem tests that we have. Could you add something like this ?
Ok, I'll write this tests tomorrow.
Quoted 6 lines
fs/f2fs/Config.in line 22
Quoted 2 lines
fs/f2fs/Config.in new file +2
-0+config BR2_TARGET_ROOTFS_F2FS_COLD_FILES + string "extension list for cold files"Any reason for having added this option for cold files, but not the symmetric option for hot files ?
I've missed hot files because manual page don't says anything about them. I'll add a patch for hot files too.
Quoted 10 lines
fs/f2fs/Config.in line 26
Quoted 4 lines
fs/f2fs/Config.in new file +4
-0+ help + Specify a file extension list in order f2fs to treat them + as cold files. The default list includes most of multimedia + file extensions such as jpg, gif, mpeg, mkv, and so on.I've seen this after committing, but the help text of mkfs.f2fs seems to imply that the list of extensions should be comma separated. If that's the case, it should be mentioned in the help text, because most Buildroot options use space-separated lists, not comma-separated ones.
Quoted 6 lines
fs/f2fs/Config.in line 29
Quoted 2 lines
fs/f2fs/Config.in new file +2
-0+config BR2_TARGET_ROOTFS_F2FS_OVERPROVISION + int "size for overprovision area (0 for auto calculation)"Actually 0 is not auto-calculation. 0 will not pass any -O option, and therefore mkfs.f2fs will use its default of 5%.
mkfs.f2fs says that default overprovision ratio is 5%, but in manual page you can read If not specified, the best number will be assigned automatically accoring to the partition size. This is a bit confusing. After reading source code I'm sure if -o option is omitted or set to 0, overprovision ratio is auto calculated.
Quoted 12 lines
Also, this option doesn't give the size, but the ratio.
I've fixed both aspects when committing.
fs/f2fs/f2fs.mk line 41
Quoted 1 line
fs/f2fs/f2fs.mk new file +1
-0+ dd if=/dev/zero of=$@ bs=1 count=0 seek=$(F2FS_SIZE)I've replaced by a truncate, as suggested by Yann.
Thanks!
Thomas