Skip to contentSkip to conversation

Create ext* filesystems using debugfs

PATCH In review Robert Yang 5 participants 22 messages Feb 26, 2013 – Feb 27, 2013

lore
Share & tools
lore
#0 lore raw

cover Create ext* filesystems using debugfs

===Contents===
* Summary
* Brief design
* Test info
* The rootfs generation time changes
* The rootfs size changes

* Summary:
The ext3 and ext4 rootfs generation relied on genext2fs and flipping some bits to "convert" the ext2 filesystem to ext3 and ext4 in the past, now we use the mkfs.ext3/ext4 to create the image, and use mkdebugfs.sh to copy the files to the image.

* Brief design:
- Upgrade the e2fsprogs to the update to date version (the git repo) which contains the "symlink" command in debugfs shell that we need.
- Replace the current genext2fs command with mkdebugfs.sh in image_types.bbclass.
- The new steps to generate the root filesystem are:

    -> use "dd" command to make rootfs.ext* file
    -> run "mkfs.ext* rootfs.ext*" to create the filesystem
    -> run the mkdebugfs.sh to generate the root filesystem

* Test info: (MACHINE = qemuarm, IMAGE_FSTYPES = "ext2 ext3 ext4 tar.bz2")

  $ bitbake core-image-minimal core-image-sato meta-toolchain-sdk core-image-sato-sdk
  $ runqemu runqemu qemuarm core-image-sato ext3
  $ runqemu runqemu qemuarm core-image-minimal ext3

All of them are OK

* The rootfs generation time changes
- For a core-image-minimal generation (IMAGE_FSTYPES="tar.bz2 ext3"):

    Before:
    $ time bitbake core-image-minimal
    
    real    1m10.823s
    user    0m37.108s
    sys     0m15.894s
    
    After:
    $ time bitbake core-image-minimal
    
    real    1m17.501s
    user    0m29.304s
    sys     0m20.731s
    
    # 7 seconds lost.

- For a core-image-sato generation (IMAGE_FSTYPES="tar.bz2 ext3"):

    Before:
    $ time bitbake core-image-sato
    
    real    11m10.645s
    user    2m43.503s
    sys     1m1.589s
    
    After:
    $ time bitbake core-image-sato
    
    real    11m53.131s
    user    3m18.988s
    sys     2m8.350s
    
    # 43 seconds lost.

* The rootfs size changes
- The image size are the same by "ls -h", but different by "du -sh":

    (core-image-sato)
  
    $ ls -lh BEFORE.rootfs.ext3 AFTER.rootfs.ext3 | awk '{print $5"\t"$NF}'
    357M BEFORE.rootfs.ext3
    357M AFTER.rootfs.ext3
    
    $ du -sh BEFORE.rootfs.ext3 AFTER.rootfs.ext3
    238M    BEFORE.rootfs.ext3
    357M    AFTER.rootfs.ext3
    
    It seems that the genext2fs has optimized the rootfs generation.

// Robert

The following changes since commit 2cc4fe4a0874c42421b1bf3fa100160a9e60a9da:

upstream_tracking.inc: Coonectivity and multimedia packages updates (2013-02-25 05:58:20 -0800)

are available in the git repository at:

git://git.pokylinux.org/poky-contrib robert/e2fsprogs
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/e2fsprogs

Robert Yang (7):
e2fsprogs: upgrade to the git version (rename only)
e2fsprogs: upgrade to the git version
e2fsprogs: add the original mkdebugfs.sh mkdebugfs.sh: convert the tab to 4 spaces mkdebugfs.sh: several fixes
e2fsprogs: ship mkfsdebug.sh image_types.bbclass: replace genext2fs with mkdebugfs.sh

 meta/classes/image_types.bbclass                   | 45 +++++-------
 meta/recipes-devtools/e2fsprogs/e2fsprogs.inc      |  4 +-
 .../{e2fsprogs-1.42.6 => e2fsprogs}/acinclude.m4   |  0
 .../fallocate.patch                                |  0
 .../e2fsprogs/e2fsprogs/mkdebugfs.sh               | 82 ++++++++++++++++++++++
 .../{e2fsprogs-1.42.6 => e2fsprogs}/mkdir.patch    |  0
 .../remove.ldconfig.call.patch                     |  0
 .../{e2fsprogs_1.42.6.bb => e2fsprogs_git.bb}      |  6 +-
 8 files changed, 106 insertions(+), 31 deletions(-)
 rename meta/recipes-devtools/e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/acinclude.m4 (100%)
 rename meta/recipes-devtools/e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/fallocate.patch (100%)
 create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs/mkdebugfs.sh
 rename meta/recipes-devtools/e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/mkdir.patch (100%)
 rename meta/recipes-devtools/e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/remove.ldconfig.call.patch (100%)
 rename meta/recipes-devtools/e2fsprogs/{e2fsprogs_1.42.6.bb => e2fsprogs_git.bb} (92%)
Signature · 1 line
1.7.11.2
#1 lore raw

PATCH 1/7 e2fsprogs: upgrade to the git version (rename only)

We will replace the "genext2fs" command in the bbclass with the debugfs command from e2fsprogs, and we need the "symlink" subcommand in the debugfs which only exists in the up-to-date git repo currently, so change the source to use the git repo.

renamed:    e2fsprogs-1.42.6/acinclude.m4 -> e2fsprogs/acinclude.m4
renamed:    e2fsprogs-1.42.6/fallocate.patch -> e2fsprogs/fallocate.patch
renamed:    e2fsprogs-1.42.6/mkdir.patch -> e2fsprogs/mkdir.patch
renamed:    e2fsprogs-1.42.6/remove.ldconfig.call.patch -> e2fsprogs/remove.ldconfig.call.patch
renamed:    e2fsprogs_1.42.6.bb -> e2fsprogs_git.bb

[YOCTO #3848]

---
 .../e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/acinclude.m4                | 0
 .../e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/fallocate.patch             | 0
 .../e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/mkdir.patch                 | 0
 .../e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/remove.ldconfig.call.patch  | 0
 meta/recipes-devtools/e2fsprogs/{e2fsprogs_1.42.6.bb => e2fsprogs_git.bb} | 0
 5 files changed, 0 insertions(+), 0 deletions(-)
 rename meta/recipes-devtools/e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/acinclude.m4 (100%)
 rename meta/recipes-devtools/e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/fallocate.patch (100%)
 rename meta/recipes-devtools/e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/mkdir.patch (100%)
 rename meta/recipes-devtools/e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/remove.ldconfig.call.patch (100%)
 rename meta/recipes-devtools/e2fsprogs/{e2fsprogs_1.42.6.bb => e2fsprogs_git.bb} (100%)
meta/recipes-devtools/e2fsprogs/e2fsprogs/acinclude.m4 renamed from meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.6/acinclude.m4 +0 -0
similarity index 100%
rename from meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.6/acinclude.m4
rename to meta/recipes-devtools/e2fsprogs/e2fsprogs/acinclude.m4
meta/recipes-devtools/e2fsprogs/e2fsprogs/fallocate.patch renamed from meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.6/fallocate.patch +0 -0
similarity index 100%
rename from meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.6/fallocate.patch
rename to meta/recipes-devtools/e2fsprogs/e2fsprogs/fallocate.patch
meta/recipes-devtools/e2fsprogs/e2fsprogs/mkdir.patch renamed from meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.6/mkdir.patch +0 -0
similarity index 100%
rename from meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.6/mkdir.patch
rename to meta/recipes-devtools/e2fsprogs/e2fsprogs/mkdir.patch
meta/recipes-devtools/e2fsprogs/e2fsprogs/remove.ldconfig.call.patch renamed from meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.6/remove.ldconfig.call.patch +0 -0
similarity index 100%
rename from meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.6/remove.ldconfig.call.patch
rename to meta/recipes-devtools/e2fsprogs/e2fsprogs/remove.ldconfig.call.patch
meta/recipes-devtools/e2fsprogs/e2fsprogs_git.bb renamed from meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.6.bb +0 -0
similarity index 100%
rename from meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.6.bb
rename to meta/recipes-devtools/e2fsprogs/e2fsprogs_git.bb
Signature · 1 line
1.7.11.2
#2 lore raw

PATCH 2/7 e2fsprogs: upgrade to the git version

In reply to Robert Yang

Change the source from tarball to git repo, and upgrade it to the up-to-date git version.

[YOCTO #3848]

---
 meta/recipes-devtools/e2fsprogs/e2fsprogs.inc    | 4 +++-
 meta/recipes-devtools/e2fsprogs/e2fsprogs_git.bb | 4 ++--
 2 files changed, 5 insertions(+), 3 deletions(-)
meta/recipes-devtools/e2fsprogs/e2fsprogs.inc +3 -1
index aea74f7..d8ae037 100644
@@ -19,8 +19,10 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=b48f21d765b875bd10400975d12c1ca2 \
 SECTION = "base"
 DEPENDS = "util-linux"
 
-SRC_URI = "${SOURCEFORGE_MIRROR}/e2fsprogs/e2fsprogs-${PV}.tar.gz \
+SRC_URI = "git://git.kernel.org/pub/scm/fs/ext2/${BPN}.git;protocol=git \
            file://mkdir.patch"
 
+S = "${WORKDIR}/git"
+
 inherit autotools gettext
 
meta/recipes-devtools/e2fsprogs/e2fsprogs_git.bb +2 -2
index e6d90a1..70255ce 100644
@@ -6,8 +6,8 @@ SRC_URI += "file://acinclude.m4 \
             file://remove.ldconfig.call.patch \
 "
 
-SRC_URI[md5sum] = "9e444c240c1001b3292d108fbad0f49c"
-SRC_URI[sha256sum] = "726b5919edeaee5b077d7a5b602284f1453ea31cf499efa433af26c679b667a5"
+SRCREV = "fca8b1b2416c384b91b4fe00d6d37cfccb8fb3d5"
+PV = "1.43-WIP+git${SRCPV}"
 
 EXTRA_OECONF += "--libdir=${base_libdir} --sbindir=${base_sbindir} --enable-elf-shlibs --disable-libuuid --disable-uuidd"
 EXTRA_OECONF_darwin = "--libdir=${base_libdir} --sbindir=${base_sbindir} --enable-bsd-shlibs"
Signature · 1 line
1.7.11.2
#3 lore raw

PATCH 3/7 e2fsprogs: add the original mkdebugfs.sh

In reply to Robert Yang

This script is from Darren Hart, it will be used for creating the extX filesystem from a given directory, which will replace the genext2fs in image_types.bbclass.

It isn't shipp in e2fsprogs package at the moment, just add it intact, and will do more work on it in the following commits.

[YOCTO #3848]

---
 .../e2fsprogs/e2fsprogs/mkdebugfs.sh               | 49 ++++++++++++++++++++++
 1 file changed, 49 insertions(+)
 create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs/mkdebugfs.sh
meta/recipes-devtools/e2fsprogs/e2fsprogs/mkdebugfs.sh new file +49 -0
new file mode 100644
index 0000000..5dfa380
@@ -0,0 +1,49 @@
+#!/bin/sh
+SRCDIR=$1
+DEVICE=$2
+DEBUGFS="debugfs/debugfs"
+
+yes | mkfs.ext4 test.img
+
+{
+	CWD="/"
+	find $SRCDIR | while read FILE; do
+		#TGT=${FILE#$SRCDIR}
+		TGT=$(basename "${FILE#$SRCDIR}")
+		DIR=$(dirname "${FILE#$SRCDIR}")
+
+		# Skip the root dir
+		if [ -z "$TGT" ]; then
+			continue
+		fi
+
+		if [ "$DIR" != "$CWD" ]; then
+			echo "cd $DIR"
+			CWD="$DIR"
+		fi
+
+		case $(stat -c "%F" $FILE) in
+		"directory")
+			echo "mkdir $TGT"
+			;;
+		"regular file")
+			echo "write $FILE $TGT"
+			;;
+		"symbolic link")
+			LINK_TGT=$(ls -l $FILE | sed -e 's/.*-> //')
+			echo "symlink $TGT $LINK_TGT"
+			;;
+		"block special file")
+			DEVNO=$(stat -c "%t %T" $FILE)
+			echo "mknod $TGT b $DEVNO"
+			;;
+		"character special file")
+			DEVNO=$(stat -c "%t %T" $FILE)
+			echo "mknod $TGT c $DEVNO"
+			;;
+		*)
+			echo "Unknown file $FILE" 1>&2
+			;;
+		esac
+	done
+} | $DEBUGFS -w -f /dev/stdin $DEVICE
Signature · 1 line
1.7.11.2
#4 lore raw

PATCH 4/7 mkdebugfs.sh: convert the tab to 4 spaces

In reply to Robert Yang

[YOCTO #3848]

---
 .../e2fsprogs/e2fsprogs/mkdebugfs.sh               | 74 +++++++++++-----------
 1 file changed, 37 insertions(+), 37 deletions(-)
meta/recipes-devtools/e2fsprogs/e2fsprogs/mkdebugfs.sh +37 -37
index 5dfa380..41a13cb 100644
@@ -6,44 +6,44 @@ DEBUGFS="debugfs/debugfs"
 yes | mkfs.ext4 test.img
 
 {
-	CWD="/"
-	find $SRCDIR | while read FILE; do
-		#TGT=${FILE#$SRCDIR}
-		TGT=$(basename "${FILE#$SRCDIR}")
-		DIR=$(dirname "${FILE#$SRCDIR}")
+    CWD="/"
+    find $SRCDIR | while read FILE; do
+        #TGT=${FILE#$SRCDIR}
+        TGT=$(basename "${FILE#$SRCDIR}")
+        DIR=$(dirname "${FILE#$SRCDIR}")
 
-		# Skip the root dir
-		if [ -z "$TGT" ]; then
-			continue
-		fi
+        # Skip the root dir
+        if [ -z "$TGT" ]; then
+            continue
+        fi
 
-		if [ "$DIR" != "$CWD" ]; then
-			echo "cd $DIR"
-			CWD="$DIR"
-		fi
+        if [ "$DIR" != "$CWD" ]; then
+            echo "cd $DIR"
+            CWD="$DIR"
+        fi
 
-		case $(stat -c "%F" $FILE) in
-		"directory")
-			echo "mkdir $TGT"
-			;;
-		"regular file")
-			echo "write $FILE $TGT"
-			;;
-		"symbolic link")
-			LINK_TGT=$(ls -l $FILE | sed -e 's/.*-> //')
-			echo "symlink $TGT $LINK_TGT"
-			;;
-		"block special file")
-			DEVNO=$(stat -c "%t %T" $FILE)
-			echo "mknod $TGT b $DEVNO"
-			;;
-		"character special file")
-			DEVNO=$(stat -c "%t %T" $FILE)
-			echo "mknod $TGT c $DEVNO"
-			;;
-		*)
-			echo "Unknown file $FILE" 1>&2
-			;;
-		esac
-	done
+        case $(stat -c "%F" $FILE) in
+        "directory")
+            echo "mkdir $TGT"
+            ;;
+        "regular file")
+            echo "write $FILE $TGT"
+            ;;
+        "symbolic link")
+            LINK_TGT=$(ls -l $FILE | sed -e 's/.*-> //')
+            echo "symlink $TGT $LINK_TGT"
+            ;;
+        "block special file")
+            DEVNO=$(stat -c "%t %T" $FILE)
+            echo "mknod $TGT b $DEVNO"
+            ;;
+        "character special file")
+            DEVNO=$(stat -c "%t %T" $FILE)
+            echo "mknod $TGT c $DEVNO"
+            ;;
+        *)
+            echo "Unknown file $FILE" 1>&2
+            ;;
+        esac
+    done
 } | $DEBUGFS -w -f /dev/stdin $DEVICE
Signature · 1 line
1.7.11.2
#5 lore raw

Re: [PATCH 4/7] mkdebugfs.sh: convert the tab to 4 spaces

And NACK on this one :-)

Shell scripts should use tabs.

--
Darren

Quoted 94 lines · Robert Yang

[YOCTO #3848]

---
 .../e2fsprogs/e2fsprogs/mkdebugfs.sh               | 74 +++++++++++-----------
 1 file changed, 37 insertions(+), 37 deletions(-)
meta/recipes-devtools/e2fsprogs/e2fsprogs/mkdebugfs.sh +37 -37
index 5dfa380..41a13cb 100644
@@ -6,44 +6,44 @@ DEBUGFS="debugfs/debugfs"
 yes | mkfs.ext4 test.img
 
 {
-	CWD="/"
-	find $SRCDIR | while read FILE; do
-		#TGT=${FILE#$SRCDIR}
-		TGT=$(basename "${FILE#$SRCDIR}")
-		DIR=$(dirname "${FILE#$SRCDIR}")
+    CWD="/"
+    find $SRCDIR | while read FILE; do
+        #TGT=${FILE#$SRCDIR}
+        TGT=$(basename "${FILE#$SRCDIR}")
+        DIR=$(dirname "${FILE#$SRCDIR}")
 
-		# Skip the root dir
-		if [ -z "$TGT" ]; then
-			continue
-		fi
+        # Skip the root dir
+        if [ -z "$TGT" ]; then
+            continue
+        fi
 
-		if [ "$DIR" != "$CWD" ]; then
-			echo "cd $DIR"
-			CWD="$DIR"
-		fi
+        if [ "$DIR" != "$CWD" ]; then
+            echo "cd $DIR"
+            CWD="$DIR"
+        fi
 
-		case $(stat -c "%F" $FILE) in
-		"directory")
-			echo "mkdir $TGT"
-			;;
-		"regular file")
-			echo "write $FILE $TGT"
-			;;
-		"symbolic link")
-			LINK_TGT=$(ls -l $FILE | sed -e 's/.*-> //')
-			echo "symlink $TGT $LINK_TGT"
-			;;
-		"block special file")
-			DEVNO=$(stat -c "%t %T" $FILE)
-			echo "mknod $TGT b $DEVNO"
-			;;
-		"character special file")
-			DEVNO=$(stat -c "%t %T" $FILE)
-			echo "mknod $TGT c $DEVNO"
-			;;
-		*)
-			echo "Unknown file $FILE" 1>&2
-			;;
-		esac
-	done
+        case $(stat -c "%F" $FILE) in
+        "directory")
+            echo "mkdir $TGT"
+            ;;
+        "regular file")
+            echo "write $FILE $TGT"
+            ;;
+        "symbolic link")
+            LINK_TGT=$(ls -l $FILE | sed -e 's/.*-> //')
+            echo "symlink $TGT $LINK_TGT"
+            ;;
+        "block special file")
+            DEVNO=$(stat -c "%t %T" $FILE)
+            echo "mknod $TGT b $DEVNO"
+            ;;
+        "character special file")
+            DEVNO=$(stat -c "%t %T" $FILE)
+            echo "mknod $TGT c $DEVNO"
+            ;;
+        *)
+            echo "Unknown file $FILE" 1>&2
+            ;;
+        esac
+    done
 } | $DEBUGFS -w -f /dev/stdin $DEVICE
Signature · 3 lines
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel
#6 lore raw

PATCH 5/7 mkdebugfs.sh: several fixes

In reply to Robert Yang

Fix mkdebugfs.sh:

* Add a simple usage
* Add checking for the number of the parameters
* Add the "regular empty file" and "fifo" file type
* Set mode, uid and gid for the file
* Save the command lines to a file and batch run them

[YOCTO #3848]

---
 .../e2fsprogs/e2fsprogs/mkdebugfs.sh               | 43 +++++++++++++++++++---
 1 file changed, 38 insertions(+), 5 deletions(-)
meta/recipes-devtools/e2fsprogs/e2fsprogs/mkdebugfs.sh +38 -5
index 41a13cb..88abc35 100644
@@ -1,10 +1,25 @@
 #!/bin/sh
+
+do_usage () {
+    cat << _EOF
+Usage: mkdebugfs.sh <source> <device>
+Create an ext2/ext3/ext4 filesystem from a directory or file
+
+  source: The source directory or file
+  device: The target device
+
+_EOF
+    exit 1
+}
+
+[ $# -ne 2 ] && do_usage
+
 SRCDIR=$1
 DEVICE=$2
-DEBUGFS="debugfs/debugfs"
-
-yes | mkfs.ext4 test.img
+DEBUGFS="debugfs"
+CMD_FILE=`mktemp`
 
+# Save the debugfs command lines to a file, then run them
 {
     CWD="/"
     find $SRCDIR | while read FILE; do
@@ -26,7 +41,7 @@ yes | mkfs.ext4 test.img
         "directory")
             echo "mkdir $TGT"
             ;;
-        "regular file")
+        "regular file" | "regular empty file")
             echo "write $FILE $TGT"
             ;;
         "symbolic link")
@@ -41,9 +56,27 @@ yes | mkfs.ext4 test.img
             DEVNO=$(stat -c "%t %T" $FILE)
             echo "mknod $TGT c $DEVNO"
             ;;
+        "fifo")
+            echo "mknod $TGT p"
+            ;;
         *)
             echo "Unknown file $FILE" 1>&2
             ;;
         esac
+
+        # Set the file mode
+        MY_MODE=$(stat -c "%f" $FILE)
+        echo "sif $TGT mode 0x$MY_MODE"
+
+        # Set uid and gid
+        MY_UID=$(stat -c "%u" $FILE)
+        echo "sif $TGT uid $MY_UID"
+        MY_GID=$(stat -c "%g" $FILE)
+        echo "sif $TGT gid $MY_GID"
     done
-} | $DEBUGFS -w -f /dev/stdin $DEVICE
+} > $CMD_FILE
+
+# Run the command lines
+$DEBUGFS -w -f $CMD_FILE $DEVICE
+
+rm -f $CMD_FILE
Signature · 1 line
1.7.11.2
#7 lore raw

Re: [PATCH 5/7] mkdebugfs.sh: several fixes

Quoted 6 lines · Robert Yang

Fix mkdebugfs.sh:

         *)
             echo "Unknown file $FILE" 1>&2
             ;;
         esac

I know this doesn't come from your work, but since you're making changes, I think it would make for a better error message if the above said something along the lines of:

    echo "Unknown/unhandled file type '$(stat -c "%F" $FILE)' file:$FILE"
#8 lore raw

Re: [PATCH 5/7] mkdebugfs.sh: several fixes

Quoted 8 lines · Trevor Woerner
Quoted 6 lines · Robert Yang

Fix mkdebugfs.sh:

         *)
             echo "Unknown file $FILE" 1>&2
             ;;
         esac
Quoted 6 lines

I know this doesn't come from your work, but since you're making changes, I think it would make for a better error message if the above said something along the lines of:

    echo "Unknown/unhandled file type '$(stat -c "%F" $FILE)' file:$FILE"

That's fine. Also, there is no need to send my script first and then patch it. It was a 5 minute hack, so just modify and include as a single patch.

Signature · 3 lines
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel
#9 lore raw

PATCH 6/7 e2fsprogs: ship mkfsdebug.sh

In reply to Robert Yang

Ship mkfsdebug.sh, it will be installed to /usr/bin/mkfsdebug.sh and used for replacing the genext2fs in image_types.bbclass.

[YOCTO #3848]

---
 meta/recipes-devtools/e2fsprogs/e2fsprogs_git.bb | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-devtools/e2fsprogs/e2fsprogs_git.bb +2 -0
index 70255ce..1609d42 100644
@@ -4,6 +4,7 @@ PR = "r0"
 
 SRC_URI += "file://acinclude.m4 \
             file://remove.ldconfig.call.patch \
+            file://mkdebugfs.sh \
 "
 
 SRCREV = "fca8b1b2416c384b91b4fe00d6d37cfccb8fb3d5"
@@ -41,6 +42,7 @@ do_install_append () {
 		mv ${D}${base_libdir}/e2initrd_helper ${D}${libdir}
 		mv ${D}${base_libdir}/pkgconfig ${D}${libdir}
 	fi
+	install -m 0755 ${WORKDIR}/mkdebugfs.sh ${D}${bindir}
 }
 
 RDEPENDS_e2fsprogs = "e2fsprogs-badblocks"
Signature · 1 line
1.7.11.2
#10 lore raw

Re: [PATCH 6/7] e2fsprogs: ship mkfsdebug.sh

This can be merged with 3/7 right?

Quoted 31 lines · Robert Yang

Ship mkfsdebug.sh, it will be installed to /usr/bin/mkfsdebug.sh and used for replacing the genext2fs in image_types.bbclass.

[YOCTO #3848]

---
 meta/recipes-devtools/e2fsprogs/e2fsprogs_git.bb | 2 ++
 1 file changed, 2 insertions(+)
meta/recipes-devtools/e2fsprogs/e2fsprogs_git.bb +2 -0
index 70255ce..1609d42 100644
@@ -4,6 +4,7 @@ PR = "r0"
 
 SRC_URI += "file://acinclude.m4 \
             file://remove.ldconfig.call.patch \
+            file://mkdebugfs.sh \
 "
 
 SRCREV = "fca8b1b2416c384b91b4fe00d6d37cfccb8fb3d5"
@@ -41,6 +42,7 @@ do_install_append () {
 		mv ${D}${base_libdir}/e2initrd_helper ${D}${libdir}
 		mv ${D}${base_libdir}/pkgconfig ${D}${libdir}
 	fi
+	install -m 0755 ${WORKDIR}/mkdebugfs.sh ${D}${bindir}
 }
 
 RDEPENDS_e2fsprogs = "e2fsprogs-badblocks"
Signature · 3 lines
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel
#11 lore raw

PATCH 7/7 image_types.bbclass: replace genext2fs with mkdebugfs.sh

In reply to Robert Yang

The ext3 and ext4 rootfs generation relied on genext2fs and flipping some bits to "convert" the ext2 filesystem to ext3 and ext4 in the past, now we use the mkfs.ext3/ext4 to create the image, and use mkdebugfs.sh to copy the files to the image.

[YOCTO #3848]

---
 meta/classes/image_types.bbclass | 45 +++++++++++++++-------------------------
 1 file changed, 17 insertions(+), 28 deletions(-)
meta/classes/image_types.bbclass +17 -28
index 6bb113d..92f85ed 100644
@@ -135,34 +135,23 @@ IMAGE_CMD_sum.jffs2 = "${IMAGE_CMD_jffs2} && sumtool -i ${DEPLOY_DIR_IMAGE}/${IM
 
 IMAGE_CMD_cramfs = "mkcramfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cramfs ${EXTRA_IMAGECMD}"
 
-IMAGE_CMD_ext2 () {
-	rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}
-	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext2
-	mv ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext2 ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2
-	rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}
-}
+oe_mkext234fs () {
+	fstype=$1
+	extra_imagecmd=""
 
-IMAGE_CMD_ext3 () {
-	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
-	tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
-}
+	if [ $# -gt 1 ]; then
+		shift
+		extra_imagecmd=$@
+	fi
 
-oe_mkext4fs () {
-	genext2fs -b $ROOTFS_SIZE -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} $1
-	tune2fs -O extents,uninit_bg,dir_index,has_journal $1
-	e2fsck -yfDC0 $1 || chk=$?
-	case $chk in
-	0|1|2)
-	    ;;
-	*)
-	    return $chk
-	    ;;
-	esac
+	dd if=/dev/zero of=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype count=$ROOTFS_SIZE bs=1k
+	yes | mkfs.$fstype $extra_imagecmd ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype
+	mkdebugfs.sh ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype
 }
 
-IMAGE_CMD_ext4 () {
-	oe_mkext4fs ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext4
-}
+IMAGE_CMD_ext2 = "oe_mkext234fs ext2 ${EXTRA_IMAGECMD}"
+IMAGE_CMD_ext3 = "oe_mkext234fs ext3 ${EXTRA_IMAGECMD}"
+IMAGE_CMD_ext4 = "oe_mkext234fs ext4 ${EXTRA_IMAGECMD}"
 
 IMAGE_CMD_btrfs () {
 	mkfs.btrfs -b `expr ${ROOTFS_SIZE} \* 1024` ${EXTRA_IMAGECMD} -r ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.btrfs
@@ -212,7 +201,7 @@ JFFS2_ENDIANNESS ?= "${@base_conditional('SITEINFO_ENDIANNESS', 'le', '--little-
 JFFS2_ERASEBLOCK ?= "0x40000"
 EXTRA_IMAGECMD_jffs2 ?= "--pad ${JFFS2_ENDIANNESS} --eraseblock=${JFFS2_ERASEBLOCK} --no-cleanmarkers"
 
-# Change these if you want default genext2fs behavior (i.e. create minimal inode number)
+# Change these if you want default mkfs behavior (i.e. create minimal inode number)
 EXTRA_IMAGECMD_ext2 ?= "-i 8192"
 EXTRA_IMAGECMD_ext3 ?= "-i 8192"
 EXTRA_IMAGECMD_ext4 ?= "-i 8192"
@@ -223,9 +212,9 @@ IMAGE_DEPENDS = ""
 IMAGE_DEPENDS_jffs2 = "mtd-utils-native"
 IMAGE_DEPENDS_sum.jffs2 = "mtd-utils-native"
 IMAGE_DEPENDS_cramfs = "cramfs-native"
-IMAGE_DEPENDS_ext2 = "genext2fs-native"
-IMAGE_DEPENDS_ext3 = "genext2fs-native e2fsprogs-native"
-IMAGE_DEPENDS_ext4 = "genext2fs-native e2fsprogs-native"
+IMAGE_DEPENDS_ext2 = "e2fsprogs-native"
+IMAGE_DEPENDS_ext3 = "e2fsprogs-native"
+IMAGE_DEPENDS_ext4 = "e2fsprogs-native"
 IMAGE_DEPENDS_btrfs = "btrfs-tools-native"
 IMAGE_DEPENDS_squashfs = "squashfs-tools-native"
 IMAGE_DEPENDS_squashfs-lzma = "squashfs-tools-native"
Signature · 1 line
1.7.11.2
#12 lore raw
In reply to Robert Yang
Quoted 3 lines · Robert Yang

* Summary:
[...]now we use the mkfs.ext3/ext4 to create the image, and use mkdebugfs.sh to copy the files to the image.

For a while this had me confused because I somehow thought you were proposing using the Linux kernel's debugfs to create filesystems (i.e. mount -t debugfs none /sys/kernel/debug). Maybe I'm still confused, but if not I think, perhaps, it would be nice to come up with a better name for this script since the name clash will probably trip others up too.

#13 lore raw
Quoted 11 lines · Trevor Woerner
Quoted 3 lines · Robert Yang

* Summary:
[...]now we use the mkfs.ext3/ext4 to create the image, and use mkdebugfs.sh to copy the files to the image.

For a while this had me confused because I somehow thought you were proposing using the Linux kernel's debugfs to create filesystems (i.e. mount -t debugfs none /sys/kernel/debug). Maybe I'm still confused, but if not I think, perhaps, it would be nice to come up with a better name for this script since the name clash will probably trip others up too.

It is confusing, unfortunately, debugfs is provided by e2fsprogs, not something of our making. If you would prefer another name for mkdebugfs.sh, that's fine. What would you propose?

Signature · 3 lines
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel
#14 lore raw
Quoted 6 lines · Darren Hart
Quoted 5 lines · Trevor Woerner
Quoted 3 lines · Robert Yang

* Summary:
[...]now we use the mkfs.ext3/ext4 to create the image, and use mkdebugfs.sh to copy the files to the image.

Quoted 7 lines
Quoted 3 lines

For a while this had me confused because I somehow thought you were proposing using the Linux kernel's debugfs to create filesystems (i.e. mount -t debugfs none /sys/kernel/debug).

It is confusing, unfortunately, debugfs is provided by e2fsprogs, not something of our making. If you would prefer another name for mkdebugfs.sh, that's fine. What would you propose?

mkextXfs.sh ?

#15 lore raw
Quoted 11 lines · Trevor Woerner
Quoted 10 lines · Darren Hart
Quoted 8 lines · Trevor Woerner
Quoted 3 lines · Robert Yang

* Summary:
[...]now we use the mkfs.ext3/ext4 to create the image, and use mkdebugfs.sh to copy the files to the image.

For a while this had me confused because I somehow thought you were proposing using the Linux kernel's debugfs to create filesystems (i.e. mount -t debugfs none /sys/kernel/debug).

Quoted 5 lines
Quoted 3 lines

It is confusing, unfortunately, debugfs is provided by e2fsprogs, not something of our making. If you would prefer another name for mkdebugfs.sh, that's fine. What would you propose?

mkextXfs.sh ?

Likely to be confused with mke2fs itself. Perhaps:

populate-extfs.sh ?

Signature · 3 lines
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel
#16 lore raw
Quoted 5 lines · Darren Hart
Quoted 1 line

mkextXfs.sh ?

Likely to be confused with mke2fs itself. Perhaps:

populate-extfs.sh ?

Sounds great.

#17 lore raw
In reply to Robert Yang
Quoted 13 lines · Robert Yang

===Contents===
* Summary
* Brief design
* Test info
* The rootfs generation time changes
* The rootfs size changes

* Summary:
The ext3 and ext4 rootfs generation relied on genext2fs and flipping some bits to "convert" the ext2 filesystem to ext3 and ext4 in the past, now we use the mkfs.ext3/ext4 to create the image, and use mkdebugfs.sh to copy the files to the image.

Quoted 4 lines

* Brief design:
- Upgrade the e2fsprogs to the update to date version (the git repo) which contains the "symlink" command in debugfs shell that we need.

This should really be an RFC

This work was intended to be a step toward incorporating it into genext2fs directly. At this point we do not want to switch to the GIT version of e2fsprogs, there is an update to e2fsprogs pending on the patch queue that addes the symlink support (crossed with your request)

Quoted 2 lines

- Replace the current genext2fs command with mkdebugfs.sh in image_types.bbclass.

The goal is to change genext2fs to support a rootfs option an not replace it with mkdebugfs.

Quoted 2 lines

- The new steps to generate the root filesystem are:

     -> use "dd" command to make rootfs.ext* file

Why the dd?

Quoted 6 lines
     -> run "mkfs.ext* rootfs.ext*" to create the filesystem
     -> run the mkdebugfs.sh to generate the root filesystem
Quoted 6 lines

* Test info: (MACHINE = qemuarm, IMAGE_FSTYPES = "ext2 ext3 ext4 tar.bz2")

   $ bitbake core-image-minimal core-image-sato meta-toolchain-sdk core-image-sato-sdk
   $ runqemu runqemu qemuarm core-image-sato ext3
   $ runqemu runqemu qemuarm core-image-minimal ext3

All of them are OK

What about booting an ext4 image, part of the goal of this was to enable ext4.

Quoted 88 lines

* The rootfs generation time changes
- For a core-image-minimal generation (IMAGE_FSTYPES="tar.bz2 ext3"):

     Before:
     $ time bitbake core-image-minimal

     real    1m10.823s
     user    0m37.108s
     sys     0m15.894s

     After:
     $ time bitbake core-image-minimal

     real    1m17.501s
     user    0m29.304s
     sys     0m20.731s

     # 7 seconds lost.
   - For a core-image-sato generation (IMAGE_FSTYPES="tar.bz2 ext3"):

     Before:
     $ time bitbake core-image-sato

     real    11m10.645s
     user    2m43.503s
     sys     1m1.589s

     After:
     $ time bitbake core-image-sato

     real    11m53.131s
     user    3m18.988s
     sys     2m8.350s

     # 43 seconds lost.

* The rootfs size changes
- The image size are the same by "ls -h", but different by "du -sh":

     (core-image-sato)

     $ ls -lh BEFORE.rootfs.ext3 AFTER.rootfs.ext3 | awk '{print $5"\t"$NF}'
     357M BEFORE.rootfs.ext3
     357M AFTER.rootfs.ext3

     $ du -sh BEFORE.rootfs.ext3 AFTER.rootfs.ext3
     238M    BEFORE.rootfs.ext3
     357M    AFTER.rootfs.ext3

     It seems that the genext2fs has optimized the rootfs generation.

// Robert

The following changes since commit 2cc4fe4a0874c42421b1bf3fa100160a9e60a9da:

   upstream_tracking.inc: Coonectivity and multimedia packages updates (2013-02-25 05:58:20 -0800)

are available in the git repository at:

git://git.pokylinux.org/poky-contrib robert/e2fsprogs
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/e2fsprogs

Robert Yang (7):
e2fsprogs: upgrade to the git version (rename only)
e2fsprogs: upgrade to the git version
e2fsprogs: add the original mkdebugfs.sh mkdebugfs.sh: convert the tab to 4 spaces mkdebugfs.sh: several fixes
e2fsprogs: ship mkfsdebug.sh image_types.bbclass: replace genext2fs with mkdebugfs.sh

  meta/classes/image_types.bbclass                   | 45 +++++-------
  meta/recipes-devtools/e2fsprogs/e2fsprogs.inc      |  4 +-
  .../{e2fsprogs-1.42.6 => e2fsprogs}/acinclude.m4   |  0
  .../fallocate.patch                                |  0
  .../e2fsprogs/e2fsprogs/mkdebugfs.sh               | 82 ++++++++++++++++++++++
  .../{e2fsprogs-1.42.6 => e2fsprogs}/mkdir.patch    |  0
  .../remove.ldconfig.call.patch                     |  0
  .../{e2fsprogs_1.42.6.bb => e2fsprogs_git.bb}      |  6 +-
  8 files changed, 106 insertions(+), 31 deletions(-)

rename meta/recipes-devtools/e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/acinclude.m4 (100%) rename meta/recipes-devtools/e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/fallocate.patch (100%) create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs/mkdebugfs.sh
rename meta/recipes-devtools/e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/mkdir.patch (100%) rename meta/recipes-devtools/e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/remove.ldconfig.call.patch (100%) rename meta/recipes-devtools/e2fsprogs/{e2fsprogs_1.42.6.bb => e2fsprogs_git.bb} (92%)

#18 lore raw
Quoted 14 lines · Saul Wold
Quoted 13 lines · Robert Yang

===Contents===
* Summary
* Brief design
* Test info
* The rootfs generation time changes
* The rootfs size changes

* Summary:
The ext3 and ext4 rootfs generation relied on genext2fs and flipping some bits to "convert" the ext2 filesystem to ext3 and ext4 in the past, now we use the mkfs.ext3/ext4 to create the image, and use mkdebugfs.sh to copy the files to the image.

Quoted 8 lines
Quoted 4 lines

* Brief design:
- Upgrade the e2fsprogs to the update to date version (the git repo) which contains the "symlink" command in debugfs shell that we need.

This should really be an RFC

This work was intended to be a step toward incorporating it into genext2fs directly.

I think that was a thinko, but I'll correct for everyone else's benefit. This will eventually be incorporated into libext2fs and mke2fs directly and genext2fs will be retired as obsolete.

Quoted 9 lines

At this point we do not want to switch to the GIT
version of e2fsprogs, there is an update to e2fsprogs pending on the patch queue that addes the symlink support (crossed with your request)

Quoted 2 lines

- Replace the current genext2fs command with mkdebugfs.sh in image_types.bbclass.

The goal is to change genext2fs to support a rootfs option an not replace it with mkdebugfs.

Again, mke2fs. We can go about this in two ways. One is to just prototype this in a branch and use it to validate the functionality and not make any changes to oe-core image generation until mke2fs has initial directory support. The other is to merge this and get broader testing of the concept and later move to the full mke2fs implementation once it becomes available. I understand the resistance to the latter, but long term I think it will result in a more robust solution as we will have caught more of the corner cases and have been able to do a better job integrating into mke2fs the first time.

Quoted 4 lines
Quoted 3 lines

- The new steps to generate the root filesystem are:

     -> use "dd" command to make rootfs.ext* file

Why the dd?

mke2fs requires an image file to format. We could create it in other ways, but this seems reasonable. We could probably reduce the time to create it by using a larger block size. Say 1M?

Quoted 12 lines
Quoted 12 lines
     -> run "mkfs.ext* rootfs.ext*" to create the filesystem
     -> run the mkdebugfs.sh to generate the root filesystem

* Test info: (MACHINE = qemuarm, IMAGE_FSTYPES = "ext2 ext3 ext4 tar.bz2")

   $ bitbake core-image-minimal core-image-sato meta-toolchain-sdk core-image-sato-sdk
   $ runqemu runqemu qemuarm core-image-sato ext3
   $ runqemu runqemu qemuarm core-image-minimal ext3
Quoted 4 lines
Quoted 1 line

All of them are OK

What about booting an ext4 image, part of the goal of this was to enable ext4.

Seconded.

Signature · 3 lines
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel
#19 lore raw
In reply to Robert Yang
Quoted 36 lines · Robert Yang
    Before:
    $ time bitbake core-image-minimal
    
    real    1m10.823s
    user    0m37.108s
    sys     0m15.894s
    
    After:
    $ time bitbake core-image-minimal
    
    real    1m17.501s
    user    0m29.304s
    sys     0m20.731s
    
    # 7 seconds lost.

- For a core-image-sato generation (IMAGE_FSTYPES="tar.bz2 ext3"):

    Before:
    $ time bitbake core-image-sato
    
    real    11m10.645s
    user    2m43.503s
    sys     1m1.589s
    
    After:
    $ time bitbake core-image-sato
    
    real    11m53.131s
    user    3m18.988s
    sys     2m8.350s
    
    # 43 seconds lost.

* The rootfs size changes
- The image size are the same by "ls -h", but different by "du -sh":

Quoted 9 lines
    (core-image-sato)
  
    $ ls -lh BEFORE.rootfs.ext3 AFTER.rootfs.ext3 | awk '{print $5"\t"$NF}'
    357M BEFORE.rootfs.ext3
    357M AFTER.rootfs.ext3
    
    $ du -sh BEFORE.rootfs.ext3 AFTER.rootfs.ext3
    238M    BEFORE.rootfs.ext3
    357M    AFTER.rootfs.ext3

So, am I understanding correctly that your patch causes the rootfs generation to take longer, and the resulting files to be bigger? It doesn't seem totally obvious to me that this is a good thing, and your cover letter doesn't provide any other explanation for why the changes are desirable. Can you expand on the rationale for doing this?

Also, when sending patches for review, there is no point in sending a series which adds a new file and then makes changes to it as separate patches (in particular when the changes are just whitespace). Please squash those into a single patch before posting them on the list.

thanks

p.

#20 lore raw
Quoted 47 lines · Phil Blundell
Quoted 36 lines · Robert Yang
    Before:
    $ time bitbake core-image-minimal
    
    real    1m10.823s
    user    0m37.108s
    sys     0m15.894s
    
    After:
    $ time bitbake core-image-minimal
    
    real    1m17.501s
    user    0m29.304s
    sys     0m20.731s
    
    # 7 seconds lost.

- For a core-image-sato generation (IMAGE_FSTYPES="tar.bz2 ext3"):

    Before:
    $ time bitbake core-image-sato
    
    real    11m10.645s
    user    2m43.503s
    sys     1m1.589s
    
    After:
    $ time bitbake core-image-sato
    
    real    11m53.131s
    user    3m18.988s
    sys     2m8.350s
    
    # 43 seconds lost.

* The rootfs size changes
- The image size are the same by "ls -h", but different by "du -sh":

Quoted 9 lines
    (core-image-sato)
  
    $ ls -lh BEFORE.rootfs.ext3 AFTER.rootfs.ext3 | awk '{print $5"\t"$NF}'
    357M BEFORE.rootfs.ext3
    357M AFTER.rootfs.ext3
    
    $ du -sh BEFORE.rootfs.ext3 AFTER.rootfs.ext3
    238M    BEFORE.rootfs.ext3
    357M    AFTER.rootfs.ext3
Quoted 5 lines

So, am I understanding correctly that your patch causes the rootfs generation to take longer, and the resulting files to be bigger? It doesn't seem totally obvious to me that this is a good thing, and your cover letter doesn't provide any other explanation for why the changes are desirable. Can you expand on the rationale for doing this?

A better motivation description is indeed required.

Basically, genext2fs doesn't support creating ext4 filesystems. It creates, as I understand it, an ext2 filesystem, then adds a journal, and sets some bits. It can't support the newer features like extents. So what we end up with is a bit of a hack for a filesystem.

The ext tools (e2fsprogs) unfortunately don't provide an integrated solution for generating prepopulated filesystem images as many other mkfs* tools do. One thing missing was symlink support in libext2fs. I added that support and demonstrated a script which uses the e2fsprogs debugfs tool that can populate the newly formatted filesystem from a directory and without root privileges.

Robert's patches integrate this stage of development into OE-Core. As I replied to Saul's 0/7 response:

"We can go about this in two ways. One is to just
prototype this in a branch and use it to validate the functionality and not make any changes to oe-core image generation until mke2fs has initial directory support. The other is to merge this and get broader testing of the concept and later move to the full mke2fs implementation once it becomes available. I understand the resistance to the latter, but long term I think it will result in a more robust solution as we will have caught more of the corner cases and have been able to do a better job integrating into mke2fs the first time."

Quoted 4 lines

Also, when sending patches for review, there is no point in sending a series which adds a new file and then makes changes to it as separate patches (in particular when the changes are just whitespace). Please squash those into a single patch before posting them on the list.

I think Robert was trying to maintain some history by keeping my original script intact. I agree, it isn't necessary and those three patches can be merged into one.

Thank you for reviewing!

Signature · 3 lines
Darren Hart
Intel Open Source Technology Center
Yocto Project - Technical Lead - Linux Kernel
#21 lore raw
In reply to Robert Yang

Hi all,

Thank you very much for all your reviewing, I will pull your comments in and send a RFC sooner.

// Robert

Quoted 118 lines · Robert Yang

===Contents===
* Summary
* Brief design
* Test info
* The rootfs generation time changes
* The rootfs size changes

* Summary:
The ext3 and ext4 rootfs generation relied on genext2fs and flipping some bits to "convert" the ext2 filesystem to ext3 and ext4 in the past, now we use the mkfs.ext3/ext4 to create the image, and use mkdebugfs.sh to copy the files to the image.

* Brief design:
- Upgrade the e2fsprogs to the update to date version (the git repo) which contains the "symlink" command in debugfs shell that we need.
- Replace the current genext2fs command with mkdebugfs.sh in image_types.bbclass.
- The new steps to generate the root filesystem are:

     -> use "dd" command to make rootfs.ext* file
     -> run "mkfs.ext* rootfs.ext*" to create the filesystem
     -> run the mkdebugfs.sh to generate the root filesystem

* Test info: (MACHINE = qemuarm, IMAGE_FSTYPES = "ext2 ext3 ext4 tar.bz2")

   $ bitbake core-image-minimal core-image-sato meta-toolchain-sdk core-image-sato-sdk
   $ runqemu runqemu qemuarm core-image-sato ext3
   $ runqemu runqemu qemuarm core-image-minimal ext3

All of them are OK

* The rootfs generation time changes
- For a core-image-minimal generation (IMAGE_FSTYPES="tar.bz2 ext3"):

     Before:
     $ time bitbake core-image-minimal

     real    1m10.823s
     user    0m37.108s
     sys     0m15.894s

     After:
     $ time bitbake core-image-minimal

     real    1m17.501s
     user    0m29.304s
     sys     0m20.731s

     # 7 seconds lost.
   - For a core-image-sato generation (IMAGE_FSTYPES="tar.bz2 ext3"):

     Before:
     $ time bitbake core-image-sato

     real    11m10.645s
     user    2m43.503s
     sys     1m1.589s

     After:
     $ time bitbake core-image-sato

     real    11m53.131s
     user    3m18.988s
     sys     2m8.350s

     # 43 seconds lost.

* The rootfs size changes
- The image size are the same by "ls -h", but different by "du -sh":

     (core-image-sato)

     $ ls -lh BEFORE.rootfs.ext3 AFTER.rootfs.ext3 | awk '{print $5"\t"$NF}'
     357M BEFORE.rootfs.ext3
     357M AFTER.rootfs.ext3

     $ du -sh BEFORE.rootfs.ext3 AFTER.rootfs.ext3
     238M    BEFORE.rootfs.ext3
     357M    AFTER.rootfs.ext3

     It seems that the genext2fs has optimized the rootfs generation.

// Robert

The following changes since commit 2cc4fe4a0874c42421b1bf3fa100160a9e60a9da:

   upstream_tracking.inc: Coonectivity and multimedia packages updates (2013-02-25 05:58:20 -0800)

are available in the git repository at:

git://git.pokylinux.org/poky-contrib robert/e2fsprogs
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/e2fsprogs

Robert Yang (7):
e2fsprogs: upgrade to the git version (rename only)
e2fsprogs: upgrade to the git version
e2fsprogs: add the original mkdebugfs.sh mkdebugfs.sh: convert the tab to 4 spaces mkdebugfs.sh: several fixes
e2fsprogs: ship mkfsdebug.sh image_types.bbclass: replace genext2fs with mkdebugfs.sh

  meta/classes/image_types.bbclass                   | 45 +++++-------
  meta/recipes-devtools/e2fsprogs/e2fsprogs.inc      |  4 +-
  .../{e2fsprogs-1.42.6 => e2fsprogs}/acinclude.m4   |  0
  .../fallocate.patch                                |  0
  .../e2fsprogs/e2fsprogs/mkdebugfs.sh               | 82 ++++++++++++++++++++++
  .../{e2fsprogs-1.42.6 => e2fsprogs}/mkdir.patch    |  0
  .../remove.ldconfig.call.patch                     |  0
  .../{e2fsprogs_1.42.6.bb => e2fsprogs_git.bb}      |  6 +-
  8 files changed, 106 insertions(+), 31 deletions(-)

rename meta/recipes-devtools/e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/acinclude.m4 (100%) rename meta/recipes-devtools/e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/fallocate.patch (100%) create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs/mkdebugfs.sh
rename meta/recipes-devtools/e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/mkdir.patch (100%) rename meta/recipes-devtools/e2fsprogs/{e2fsprogs-1.42.6 => e2fsprogs}/remove.ldconfig.call.patch (100%) rename meta/recipes-devtools/e2fsprogs/{e2fsprogs_1.42.6.bb => e2fsprogs_git.bb} (92%)