fs/ext2: misc enhancements (branch yem/fs-vol-id)

PATCH Superseded Yann E. MORIN 3 participants 9 messages Dec 6, 2014 ext2 file system

lore

Share & tools

lore

Yann E. MORIN

author

2014-12-06 12:29 UTC #0 lore raw

v3 cover fs/ext2: misc enhancements (branch yem/fs-vol-id)

Hello All!

This series enhances the fs/ext2 generation. Not much to say beside the individual commit logs... ;-)

Changes v2 -> v3:
- fix quoting for labels with a space in them (K?roly)

Changes v1 -> v2:
- move our genext2fs.sh wrapper and make it a proper host package, with extensive tweaks to the code (Thomas)

Regards,
Yann E. MORIN.

The following changes since commit c5f0e93d2cc6107cce249e1f11158aeccb4a3449:

rp-pppoe: install default conf file (2014-12-03 23:26:21 +0100)

are available in the git repository at:

git://git.busybox.net/~ymorin/git/buildroot yem/fs-vol-id

for you to fetch changes up to 0244e2c2f1c3933e110df8478544efafe66c7e5c:

fs/ext2: add option to specify a filesystem label (2014-12-06 13:26:58 +0100)

----------------------------------------------------------------
Yann E. MORIN (3):
package/mke2img: new package
fs/ext2: use the new mke2img host package fs/ext2: add option to specify a filesystem label

 fs/ext2/Config.in              |   6 +-
 fs/ext2/ext2.mk                |  19 ++++--
 fs/ext2/genext2fs.sh           | 121 ---------------------------------
 package/Config.in.host         |   1 +
 package/mke2img/Config.in.host |   9 +++
 package/mke2img/mke2img        | 148 +++++++++++++++++++++++++++++++++++++++++
 package/mke2img/mke2img.mk     |  14 ++++
 7 files changed, 188 insertions(+), 130 deletions(-)
 delete mode 100755 fs/ext2/genext2fs.sh
 create mode 100644 package/mke2img/Config.in.host
 create mode 100755 package/mke2img/mke2img
 create mode 100644 package/mke2img/mke2img.mk

Signature · 6 lines

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Yann E. MORIN

author

2014-12-06 12:29 UTC #1 lore raw

PATCH v3 1/3 package/mke2img: new package

Currently, we are using a shell script called genext2fs, that impersonates the real genext2fs. But that script does much more than just call genextfs: it also calls tune2fs and e2fsck.

Because it impersonates genext2fs, we can not easily add new options, and are constrained by the genext2fs options.

But it turns out that, of all the options supported by the real genext2fs, we only really care for a subset, namely:
- number of blocks
- number of inodes
- percentage of blocks reeserved to root
- the root directory which to generate the image from

So, we introduce a new host package, mke2img, that is intended to eventually replace genext2fs.sh.

This new script is highly modeled from the existing genext2fs.sh, but was slightly refreshed, and a new, supposedly sane set of options has been choosen for the features we need (see above), and some new options were added, too, rather than relying on the arguments order or environment variables:

  -b <nb-blocks>    number of blocks in the filesystem
  -i <nb-inodes>    number of inodes in the filesystem
  -r <pc-reserved>  percentage of reserved blocks
* -d <root-dir>     directory containing the root of the filesystem
* -o <img-file>     output image file
* -G <ext-gen>      extfs generation: 2, 3, or 4
* -R <ext-rev>      ext2 revision: 0 or 1 (1 is assumed for ext3 and ext4)
  -l <label>        filesystem label
  -u <uid>          filesystem UUID; if not specified, a random one is used

* Mandatory options

Since the upstream e2fsprogs are expected to release a new mke2fs that will be able to generate a filesystem image from a directory, we then will be able to replace all the logic in mke2img, to use mke2fs instead of the (relatively fragile) combination of the three tools we currently use.

An entry is added for it in the "Host utilities" menu, so it can be selected for use by post-{build,image} scripts. The ext2 filesystem selection is changed to select that now.

Signed-off-by "Yann E. MORIN" <[email protected]>

Cc Thomas Petazzoni <[email protected]>

Tested-by Karoly Kasza <[email protected]>

Reviewed-by Karoly Kasza <[email protected]>

---
 fs/ext2/Config.in              |   3 +-
 package/Config.in.host         |   1 +
 package/mke2img/Config.in.host |   9 +++
 package/mke2img/mke2img        | 148 +++++++++++++++++++++++++++++++++++++++++
 package/mke2img/mke2img.mk     |  14 ++++
 5 files changed, 173 insertions(+), 2 deletions(-)
 create mode 100644 package/mke2img/Config.in.host
 create mode 100755 package/mke2img/mke2img
 create mode 100644 package/mke2img/mke2img.mk

fs/ext2/Config.in +1 -2

index adba6f3..8751b62 100644
@@ -1,7 +1,6 @@
 config BR2_TARGET_ROOTFS_EXT2
 	bool "ext2/3/4 root filesystem"
-	select BR2_PACKAGE_HOST_GENEXT2FS
-	select BR2_PACKAGE_HOST_E2FSPROGS
+	select BR2_PACKAGE_HOST_MKE2IMG
 	help
 	  Build an ext2/3/4 root filesystem

package/Config.in.host +1 -0

index 185dcbc..0b7bc36 100644
@@ -10,6 +10,7 @@ menu "Host utilities"
 	source "package/genimage/Config.in.host"
 	source "package/genpart/Config.in.host"
 	source "package/lpc3250loader/Config.in.host"
+	source "package/mke2img/Config.in.host"
 	source "package/mtd/Config.in.host"
 	source "package/mtools/Config.in.host"
 	source "package/omap-u-boot-utils/Config.in.host"

package/mke2img/Config.in.host new file +9 -0

new file mode 100644
index 0000000..9781c6a
@@ -0,0 +1,9 @@
+config BR2_PACKAGE_HOST_MKE2IMG
+	bool "host mke2img"
+	select BR2_PACKAGE_HOST_E2FSPROGS
+	select BR2_PACKAGE_HOST_GENEXT2FS
+	help
+	  Easily create filesystems of the extend familly: ext2/3/4.
+
+	  This tool is bundled by, and specific to Buildroot. However, it can
+	  be used from post-images scripts is needed.

package/mke2img/mke2img new file +148 -0

148 lines not on this page · raw message

package/mke2img/mke2img.mk new file +14 -0

new file mode 100644
index 0000000..04aaa8f
@@ -0,0 +1,14 @@
+################################################################################
+#
+# mke2img
+#
+################################################################################
+
+HOST_MKE2IMG_SOURCE =
+HOST_MKE2IMG_DEPENDENCIES = host-genext2fs host-e2fsprogs
+
+define HOST_MKE2IMG_INSTALL_CMDS
+	$(INSTALL) -D -m 0755 package/mke2img/mke2img $(HOST_DIR)/usr/bin/mke2img
+endef
+
+$(eval $(host-generic-package))

Signature · 1 line

1.9.1

Thomas Petazzoni

2014-12-06 18:20 UTC #2 lore raw

PATCH v3 1/3 package/mke2img: new package

Dear Yann E. MORIN,

Quoted 15 lines · Yann E. MORIN

This new script is highly modeled from the existing genext2fs.sh, but was slightly refreshed, and a new, supposedly sane set of options has been choosen for the features we need (see above), and some new options were added, too, rather than relying on the arguments order or environment variables:
  -b <nb-blocks>    number of blocks in the filesystem
  -i <nb-inodes>    number of inodes in the filesystem
  -r <pc-reserved>  percentage of reserved blocks
* -d <root-dir>     directory containing the root of the filesystem
* -o <img-file>     output image file
* -G <ext-gen>      extfs generation: 2, 3, or 4
* -R <ext-rev>      ext2 revision: 0 or 1 (1 is assumed for ext3 and ext4)
  -l <label>        filesystem label
  -u <uid>          filesystem UUID; if not specified, a random one is used

Quoted 1 line

* Mandatory options

What about putting this in a "help" of the mke2img tool, or at least in the comment header at the beginning of the script?

package/mke2img/mke2img line 99

Quoted 2 lines

package/mke2img/mke2img new file +2 -0

+    # genext2fs does not generate a UUID, but fsck will whine if one is
+    # is missing, so we need to add a UUID.

duplicate "is".

Quoted 1 line

+ # Of course, this has to happend _before_ we run fsck.

happend -> happen

package/mke2img/mke2img line 105

Quoted 5 lines

package/mke2img/mke2img new file +5 -0

+    # Also, some ext4 metadata are based on the UUID, so we must
+    # set it before we can convert the filesystem to ext4.
+    # If the user did not specify a UUID, we generate a random one.
+    # Although a random UUID may seem bad for reproducibility, there
+    # already are so many things that are not reproducible in a

s/already are/are already/

Other than that, looks good.

Tested-by Thomas Petazzoni <[email protected]>

(on x86-64 in qemu, tested an ext4 filesystem with only the ext4 kernel driver compiled in, and then tested an ext2 filesystem with only the ext2 kernel driver compiled in).

Thanks!

Thomas

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Yann E. MORIN

author

2014-12-06 19:26 UTC #3 lore raw

PATCH v3 1/3 package/mke2img: new package

Thomas, All,

On 2014-12-06 19:20 +0100, Thomas Petazzoni spake thusly:

Quoted 16 lines · Yann E. MORIN

Quoted 15 lines
This new script is highly modeled from the existing genext2fs.sh, but was slightly refreshed, and a new, supposedly sane set of options has been choosen for the features we need (see above), and some new options were added, too, rather than relying on the arguments order or environment variables:
  -b <nb-blocks>    number of blocks in the filesystem
  -i <nb-inodes>    number of inodes in the filesystem
  -r <pc-reserved>  percentage of reserved blocks
* -d <root-dir>     directory containing the root of the filesystem
* -o <img-file>     output image file
* -G <ext-gen>      extfs generation: 2, 3, or 4
* -R <ext-rev>      ext2 revision: 0 or 1 (1 is assumed for ext3 and ext4)
  -l <label>        filesystem label
  -u <uid>          filesystem UUID; if not specified, a random one is used

Quoted 4 lines

Quoted 1 line
* Mandatory options
What about putting this in a "help" of the mke2img tool, or at least in the comment header at the beginning of the script?

Ack.

Quoted 4 lines

package/mke2img/mke2img line 99
Quoted 2 lines
package/mke2img/mke2img new file +2 -0
+    # genext2fs does not generate a UUID, but fsck will whine if one is
+    # is missing, so we need to add a UUID.
duplicate "is".

Ack.

Quoted 3 lines

Quoted 1 line
+ # Of course, this has to happend _before_ we run fsck.
happend -> happen

Ack.

Quoted 7 lines

package/mke2img/mke2img line 105

Quoted 5 lines

package/mke2img/mke2img new file +5 -0

+    # Also, some ext4 metadata are based on the UUID, so we must
+    # set it before we can convert the filesystem to ext4.
+    # If the user did not specify a UUID, we generate a random one.
+    # Although a random UUID may seem bad for reproducibility, there
+    # already are so many things that are not reproducible in a

s/already are/are already/

Nak. Positioning adverbs before the verb is a way emphasise the effects of the adverb. See for instance:
https://learnenglish.britishcouncil.org/en/english-grammar/adverbials/where-they-go-sentence

[...]
But if we want to emphasise an adverbial we can put it at the beginning of a clause
[...]

Quoted 1 line

Tested-by Thomas Petazzoni <[email protected]>

Thanks! I'll fix and respin later.

Regards,
Yann E. MORIN.

Signature · 6 lines

.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 223 225 172 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'

Yann E. MORIN

author

2014-12-06 12:29 UTC #4 lore raw

PATCH v3 2/3 fs/ext2: use the new mke2img host package

In reply to Yann E. MORIN

And get rid of the old genext2fs.sh wrapper.

Signed-off-by "Yann E. MORIN" <[email protected]>

Cc Thomas Petazzoni <[email protected]>

Tested-by Karoly Kasza <[email protected]>

Reviewed-by Karoly Kasza <[email protected]>

---
 fs/ext2/ext2.mk      |  13 +++---
 fs/ext2/genext2fs.sh | 121 ---------------------------------------------------
 2 files changed, 5 insertions(+), 129 deletions(-)
 delete mode 100755 fs/ext2/genext2fs.sh

fs/ext2/ext2.mk +5 -8

index 425f7bf..96b7a4a 100644
@@ -4,27 +4,24 @@
 #
 ################################################################################
 
-EXT2_OPTS :=
+EXT2_OPTS = -G $(BR2_TARGET_ROOTFS_EXT2_GEN) -R $(BR2_TARGET_ROOTFS_EXT2_REV)
 
 ifneq ($(strip $(BR2_TARGET_ROOTFS_EXT2_BLOCKS)),0)
 EXT2_OPTS += -b $(BR2_TARGET_ROOTFS_EXT2_BLOCKS)
 endif
 
 ifneq ($(strip $(BR2_TARGET_ROOTFS_EXT2_INODES)),0)
-EXT2_OPTS += -N $(BR2_TARGET_ROOTFS_EXT2_INODES)
+EXT2_OPTS += -i $(BR2_TARGET_ROOTFS_EXT2_INODES)
 endif
 
 ifneq ($(strip $(BR2_TARGET_ROOTFS_EXT2_RESBLKS)),0)
-EXT2_OPTS += -m $(BR2_TARGET_ROOTFS_EXT2_RESBLKS)
+EXT2_OPTS += -r $(BR2_TARGET_ROOTFS_EXT2_RESBLKS)
 endif
 
-ROOTFS_EXT2_DEPENDENCIES = host-genext2fs host-e2fsprogs
-
-EXT2_ENV = GEN=$(BR2_TARGET_ROOTFS_EXT2_GEN)
-EXT2_ENV += REV=$(BR2_TARGET_ROOTFS_EXT2_REV)
+ROOTFS_EXT2_DEPENDENCIES = host-mke2img
 
 define ROOTFS_EXT2_CMD
-	PATH=$(BR_PATH) $(EXT2_ENV) fs/ext2/genext2fs.sh -d $(TARGET_DIR) $(EXT2_OPTS) $@
+	PATH=$(BR_PATH) mke2img -d $(TARGET_DIR) $(EXT2_OPTS) -o $@
 endef
 
 rootfs-ext2-symlink:

fs/ext2/genext2fs.sh deleted +0 ~~-121~~

121 lines not on this page · raw message

Signature · 1 line

1.9.1

Thomas Petazzoni

2014-12-06 18:21 UTC #5 lore raw

PATCH v3 2/3 fs/ext2: use the new mke2img host package

Dear Yann E. MORIN,

Quoted 11 lines · Yann E. MORIN

And get rid of the old genext2fs.sh wrapper.
Signed-off-by "Yann E. MORIN" <[email protected]>
Cc Thomas Petazzoni <[email protected]>
Tested-by Karoly Kasza <[email protected]>
Reviewed-by Karoly Kasza <[email protected]>
---
 fs/ext2/ext2.mk      |  13 +++---
 fs/ext2/genext2fs.sh | 121 ---------------------------------------------------
 2 files changed, 5 insertions(+), 129 deletions(-)
 delete mode 100755 fs/ext2/genext2fs.sh

Reviewed-by Thomas Petazzoni <[email protected]>

Tested-by Thomas Petazzoni <[email protected]>

(on x86-64 in qemu, tested an ext4 filesystem with only the ext4 kernel driver compiled in, and then tested an ext2 filesystem with only the ext2 kernel driver compiled in).

Thanks,

Thomas

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

Yann E. MORIN

author

2014-12-06 12:29 UTC #6 lore raw

PATCH v3 3/3 fs/ext2: add option to specify a filesystem label

In reply to Yann E. MORIN

Filesystems of the ext familly can carry a filesystem label. Add an option for the user to specify such a label.

Signed-off-by "Yann E. MORIN" <[email protected]>

Cc Thomas Petazzoni <[email protected]>

Cc K?roly Kasza <[email protected]>

---
Changes v2 -> v3:
- fix quoting for when the label contains a space (K?roly)

---
 fs/ext2/Config.in | 3 +++
 fs/ext2/ext2.mk   | 8 ++++++++
 2 files changed, 11 insertions(+)

fs/ext2/Config.in +3 -0

index 8751b62..5b1cd0c 100644
@@ -41,6 +41,9 @@ config BR2_TARGET_ROOTFS_EXT2_REV
 	default 0   if BR2_TARGET_ROOTFS_EXT2_2r0
 	default 1   if !BR2_TARGET_ROOTFS_EXT2_2r0
 
+config BR2_TARGET_ROOTFS_EXT2_LABEL
+	string "filesystem label"
+
 config BR2_TARGET_ROOTFS_EXT2_BLOCKS
 	int "size in blocks (leave at 0 for auto calculation)"
 	default 0

fs/ext2/ext2.mk +8 -0

index 96b7a4a..1cac72e 100644
@@ -18,6 +18,14 @@ ifneq ($(strip $(BR2_TARGET_ROOTFS_EXT2_RESBLKS)),0)
 EXT2_OPTS += -r $(BR2_TARGET_ROOTFS_EXT2_RESBLKS)
 endif
 
+# Not qstrip-ing the variable, because it may contain spaces, but we must
+# qstrip it when checking. Furthermore, we need to further quote it, so
+# that the quotes do not get eaten by the echo statement when creating the
+# fakeroot script
+ifneq ($(call qstrip,$(BR2_TARGET_ROOTFS_EXT2_LABEL)),)
+EXT2_OPTS += -l '$(BR2_TARGET_ROOTFS_EXT2_LABEL)'
+endif
+
 ROOTFS_EXT2_DEPENDENCIES = host-mke2img
 
 define ROOTFS_EXT2_CMD

Signature · 1 line

1.9.1

Károly Kasza

2014-12-06 14:15 UTC #7 lore raw

PATCH v3 3/3 fs/ext2: add option to specify a filesystem label

Hi Yann, Thomas, list,

Signed-off-by "Yann E. MORIN" <[email protected]>

Quoted 3 lines

Cc Thomas Petazzoni <[email protected]>
Cc K?roly Kasza <[email protected]>

Tested-by Karoly Kasza <[email protected]>

Reviewed-by Karoly Kasza <[email protected]>

Tested the whole series of 3 together (v3), successfully built different rootfs.ext images and booted them.
Tested with master branch, x86_64 arch in QEMU, internal toolchain GCC 4.9.2 w/ uclibc, Linux 3.17.4.

Regards,
Karoly
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.busybox.net/pipermail/buildroot/attachments/20141206/dd6f609b/attachment.html>

Thomas Petazzoni

2014-12-06 18:23 UTC #8 lore raw

PATCH v3 3/3 fs/ext2: add option to specify a filesystem label

In reply to Yann E. MORIN

Dear Yann E. MORIN,

Quoted 6 lines · Yann E. MORIN

Filesystems of the ext familly can carry a filesystem label. Add an option for the user to specify such a label.
Signed-off-by "Yann E. MORIN" <[email protected]>
Cc Thomas Petazzoni <[email protected]>
Cc K?roly Kasza <[email protected]>

Reviewed-by Thomas Petazzoni <[email protected]>

Tested-by Thomas Petazzoni <[email protected]>

(on x86-64 in qemu, tested an ext4 filesystem with only the ext4 kernel driver compiled in, and then tested an ext2 filesystem with only the ext2 kernel driver compiled in).

Thanks,

Thomas

Signature · 3 lines

Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com