ext2/3/4 image generation optimization
cover ext2/3/4 image generation optimization
At Wind River we've been using and testing this change for the last couple of months. I'm not sure it's ready for inclusion ues in oe-core, but the patches have been sent to the ext4 maintainer. (Robert can further clarify if there are any questions.)
Robert Yang (12):
e2fsprogs/mke2fs: add an option: -d root-directory e2fsprogs/mke2fs: misc/util.c: implement populate_fs() e2fsprogs/mke2fs: create special file
e2fsprogs/mke2fs: create symlink
e2fsprogs/mke2fs: copy regular file
e2fsprogs/mke2fs: create directory
e2fsprogs/mke2fs: set more information for inode e2fsprogs/mke2fs: handle hardlinks
e2fsprogs/mke2fs: update the manual for the -d option e2fsprogs/mke2fs: debugfs: use the functions in misc/util.c (re-add
after revert)
e2fsprogs: enable the "-d" related patches image_types.bbclass: use mke2fs -d to create the ext2/3/4 image
meta/classes/image_types.bbclass | 3 +- ...1-mke2fs.c-add-an-option-d-root-directory.patch | 206 ++++++++++ .../0002-misc-util.c-implement-populate_fs.patch | 120 ++++++ .../0003-misc-util.c-create-special-file.patch | 98 +++++ .../0004-misc-util.c-create-symlink.patch | 62 +++ .../0005-misc-util.c-copy-regular-file.patch | 221 ++++++++++ .../0006-misc-util.c-create-directory.patch | 62 +++ ...isc-util.c-set-more-information-for-inode.patch | 79 ++++ .../0008-misc-util.c-handle-hardlinks.patch | 145 +++++++ ...s.8.in-update-the-manual-for-the-d-option.patch | 40 ++ ...-debugfs-use-the-functions-in-misc-util.c.patch | 449 +++++++++++++++++++++ .../recipes-devtools/e2fsprogs/e2fsprogs_1.42.8.bb | 10 + 12 files changed, 1493 insertions(+), 2 deletions(-) create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0001-mke2fs.c-add-an-option-d-root-directory.patch create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0002-misc-util.c-implement-populate_fs.patch create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0003-misc-util.c-create-special-file.patch create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0004-misc-util.c-create-symlink.patch create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0005-misc-util.c-copy-regular-file.patch create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0006-misc-util.c-create-directory.patch create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0007-misc-util.c-set-more-information-for-inode.patch create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0008-misc-util.c-handle-hardlinks.patch create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0009-mke2fs.8.in-update-the-manual-for-the-d-option.patch create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0010-debugfs-use-the-functions-in-misc-util.c.patch
Signature · 1 line
1.8.1.2.545.g2f19ada
RFC PATCH 1/12 e2fsprogs/mke2fs: add an option: -d root-directory
From: Robert Yang <[email protected]>
This option is used for adding the files from the root-directory to the filesystem, it is similiar to genext2fs, but genext2fs doesn't fully support ext4.
This commit describes the skeleton of the implementation:
* The "struct hdlink_s" will be used for saving hard links, I referred this from the genext2fs.
* The do_xxx_internal will be used by both mke2fs and debugfs, most of their operations are similar.
* Fix the location of ext2fs_close() to make the "make check" succeed.
--- ...1-mke2fs.c-add-an-option-d-root-directory.patch | 206 +++++++++++++++++++++ 1 file changed, 206 insertions(+) create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0001-mke2fs.c-add-an-option-d-root-directory.patch
meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0001-mke2fs.c-add-an-option-d-root-directory.patch new file +206 -0
@@ -0,0 +1,206 @@ +From d1297281c1955a6159129bb111b0a70c93f88c7c Mon Sep 17 00:00:00 2001 +From: Robert Yang <[email protected]> +Date: Sat, 9 Nov 2013 21:51:09 +0800 +Subject: [PATCH] mke2fs.c: add an option: -d root-directory + +This option is used for adding the files from the root-directory to the +filesystem, it is similiar to genext2fs, but genext2fs doesn't fully +support ext4. + +This commit describes the skeleton of the implementation: +* The "struct hdlink_s" will be used for saving hard links, I + referred this from the genext2fs. + +* The do_xxx_internal will be used by both mke2fs and debugfs, most of + their operations are similar. + +Signed-off-by: Robert Yang <[email protected]> +--- + misc/mke2fs.c | 35 +++++++++++++++++++++++++++++++---- + misc/util.c | 35 +++++++++++++++++++++++++++++++++++ + misc/util.h | 32 ++++++++++++++++++++++++++++++++ + 3 files changed, 98 insertions(+), 4 deletions(-) + +diff --git a/misc/mke2fs.c b/misc/mke2fs.c +index 22c2815..75abf83 100644 +--- a/misc/mke2fs.c ++++ b/misc/mke2fs.c +@@ -44,8 +44,6 @@ extern int optind; + #include <errno.h> + #endif + #include <sys/ioctl.h> +-#include <sys/types.h> +-#include <sys/stat.h> + #include <libgen.h> + #include <limits.h> + #include <blkid/blkid.h> +@@ -105,6 +103,7 @@ char *mount_dir; + char *journal_device; + int sync_kludge; /* Set using the MKE2FS_SYNC env. option */ + char **fs_types; ++const char *root_dir; /* Copy files from the specified directory */ + + profile_t profile; + +@@ -116,7 +115,8 @@ static void usage(void) + fprintf(stderr, _("Usage: %s [-c|-l filename] [-b block-size] " + "[-C cluster-size]\n\t[-i bytes-per-inode] [-I inode-size] " + "[-J journal-options]\n" +- "\t[-G flex-group-size] [-N number-of-inodes]\n" ++ "\t[-G flex-group-size] [-N number-of-inodes] " ++ "[-d root-directory]\n" + "\t[-m reserved-blocks-percentage] [-o creator-os]\n" + "\t[-g blocks-per-group] [-L volume-label] " + "[-M last-mounted-directory]\n\t[-O feature[,...]] " +@@ -1346,7 +1346,7 @@ profile_error: + } + + while ((c = getopt (argc, argv, +- "b:cg:i:jl:m:no:qr:s:t:vC:DE:FG:I:J:KL:M:N:O:R:ST:U:V")) != EOF) { ++ "b:cg:i:jl:m:no:qr:s:t:d:vC:DE:FG:I:J:KL:M:N:O:R:ST:U:V")) != EOF) { + switch (c) { + case 'b': + blocksize = parse_num_blocks2(optarg, -1); +@@ -1532,6 +1532,9 @@ profile_error: + case 'U': + fs_uuid = optarg; + break; ++ case 'd': ++ root_dir = optarg; ++ break; + case 'v': + verbose = 1; + break; +@@ -2665,6 +2668,30 @@ no_journal: + EXT4_FEATURE_RO_COMPAT_QUOTA)) + create_quota_inodes(fs); + ++ /* Copy files from the specified directory */ ++ if (root_dir) { ++ if (!quiet) ++ printf(_("Copying files into the device...\n")); ++ ++ /* ++ * Allocate memory for the hardlinks, we don't need free() ++ * since the lifespan will be over after the fs populated. ++ */ ++ if ((hdlinks.hdl = (struct hdlink_s *) ++ malloc(hdlink_cnt * sizeof(struct hdlink_s))) == NULL) { ++ fprintf(stderr, _("\nNot enough memory")); ++ retval = ext2fs_close(fs); ++ return retval; ++ } ++ ++ current_fs = fs; ++ root = EXT2_ROOT_INO; ++ retval = populate_fs(root, root_dir); ++ if (retval) ++ fprintf(stderr, ++ _("\nError while populating %s"), root_dir); ++ } ++ + if (!quiet) + printf(_("Writing superblocks and " + "filesystem accounting information: ")); +diff --git a/misc/util.c b/misc/util.c +index 6c93e1c..cbc7cc0 100644 +--- a/misc/util.c ++++ b/misc/util.c +@@ -32,8 +32,18 @@ + #include "ext2fs/ext2fs.h" + #include "nls-enable.h" + #include "blkid/blkid.h" ++ ++#include <fcntl.h> ++ + #include "util.h" + ++int journal_size; ++int journal_flags; ++char *journal_device; ++ ++/* For saving the hard links */ ++int hdlink_cnt = HDLINK_CNT; ++ + #ifndef HAVE_STRCASECMP + int strcasecmp (char *s1, char *s2) + { +@@ -303,3 +313,28 @@ void dump_mmp_msg(struct mmp_struct *mmp, const char *msg) + ctime(&t), mmp->mmp_nodename, mmp->mmp_bdevname); + } + } ++ ++/* Make a special file which is block, character and fifo */ ++errcode_t do_mknod_internal(ext2_ino_t cwd, const char *name, struct stat *st) ++{ ++} ++ ++/* Make a symlink name -> target */ ++errcode_t do_symlink_internal(ext2_ino_t cwd, const char *name, char *target) ++{ ++} ++ ++/* Make a directory in the fs */ ++errcode_t do_mkdir_internal(ext2_ino_t cwd, const char *name, struct stat *st) ++{ ++} ++ ++/* Copy the native file to the fs */ ++errcode_t do_write_internal(ext2_ino_t cwd, const char *src, const char *dest) ++{ ++} ++ ++/* Copy files from source_dir to fs */ ++errcode_t populate_fs(ext2_ino_t parent_ino, const char *source_dir) ++{ ++} +diff --git a/misc/util.h b/misc/util.h +index f872c38..e71caf0 100644 +--- a/misc/util.h ++++ b/misc/util.h +@@ -14,6 +14,31 @@ extern int journal_size; + extern int journal_flags; + extern char *journal_device; + ++/* For struct stat */ ++#include <sys/types.h> ++#include <sys/stat.h> ++ ++struct hdlink_s ++{ ++ ext2_ino_t src_ino; ++ ext2_ino_t dst_ino; ++}; ++ ++struct hdlinks_s ++{ ++ int count; ++ struct hdlink_s *hdl; ++}; ++ ++struct hdlinks_s hdlinks; ++ ++ext2_filsys current_fs; ++ext2_ino_t root; ++ ++/* For saving the hard links */ ++#define HDLINK_CNT 4 ++extern int hdlink_cnt; ++ + #ifndef HAVE_STRCASECMP + extern int strcasecmp (char *s1, char *s2); + #endif +@@ -25,3 +50,10 @@ extern void check_mount(const char *device, int force, const char *type); + extern unsigned int figure_journal_size(int size, ext2_filsys fs); + extern void print_check_message(int, unsigned int); + extern void dump_mmp_msg(struct mmp_struct *mmp, const char *msg); ++ ++/* For populating the filesystem */ ++extern errcode_t populate_fs(ext2_ino_t parent_ino, const char *source_dir); ++extern errcode_t do_mknod_internal(ext2_ino_t cwd, const char *name, struct stat *st); ++extern errcode_t do_symlink_internal(ext2_ino_t cwd, const char *name, char *target); ++extern errcode_t do_mkdir_internal(ext2_ino_t cwd, const char *name, struct stat *st); ++extern errcode_t do_write_internal(ext2_ino_t cwd, const char *src, const char *dest); +-- +1.8.3.1 +
Signature · 1 line
1.8.1.2.545.g2f19ada
RFC PATCH 2/12 e2fsprogs/mke2fs: misc/util.c: implement populate_fs()
In reply to Mark HatleFrom: Robert Yang <[email protected]>
Use opendir() and readdir() to read the native directory, then use lstat() to identify the file type and call the corresponding function to add the file to the filesystem, call the populate_fs() recursively if it is a directory.
NOTE: the libext2fs doesn't support create the socket file at them moment.
--- .../0002-misc-util.c-implement-populate_fs.patch | 120 +++++++++++++++++++++ 1 file changed, 120 insertions(+) create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0002-misc-util.c-implement-populate_fs.patch
meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0002-misc-util.c-implement-populate_fs.patch new file +120 -0
@@ -0,0 +1,120 @@ +From c025353bd7567b35162b6193fb0b152b1c733bcd Mon Sep 17 00:00:00 2001 +From: Robert Yang <[email protected]> +Date: Tue, 27 Aug 2013 16:12:38 +0800 +Subject: [rootdir 02/10] misc/util.c: implement populate_fs() + +Use opendir() and readdir() to read the native directory, then use +lstat() to identify the file type and call the corresponding function to +add the file to the filesystem, call the populate_fs() recursively if it +is a directory. + +NOTE: the libext2fs doesn't support create the socket file at them +moment. + +Signed-off-by: Robert Yang <[email protected]> +--- + misc/util.c | 90 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 90 insertions(+) + +diff --git a/misc/util.c b/misc/util.c +index cbc7cc0..6d87393 100644 +--- a/misc/util.c ++++ b/misc/util.c +@@ -337,4 +337,94 @@ errcode_t do_write_internal(ext2_ino_t cwd, const char *src, const char *dest) + /* Copy files from source_dir to fs */ + errcode_t populate_fs(ext2_ino_t parent_ino, const char *source_dir) + { ++ const char *name; ++ DIR *dh; ++ struct dirent *dent; ++ struct stat st; ++ char ln_target[PATH_MAX]; ++ char *func_name = "populate_fs"; ++ ext2_ino_t ino; ++ errcode_t retval; ++ int read_cnt; ++ ++ root = EXT2_ROOT_INO; ++ ++ if (chdir(source_dir) < 0) { ++ com_err(func_name, errno, ++ _("while changing working directory to \"%s\""), source_dir); ++ return errno; ++ } ++ ++ if (!(dh = opendir("."))) { ++ com_err(func_name, errno, ++ _("while openning directory \"%s\""), source_dir); ++ return errno; ++ } ++ ++ while((dent = readdir(dh))) { ++ if((!strcmp(dent->d_name, ".")) || (!strcmp(dent->d_name, ".."))) ++ continue; ++ lstat(dent->d_name, &st); ++ name = dent->d_name; ++ ++ switch(st.st_mode & S_IFMT) { ++ case S_IFCHR: ++ case S_IFBLK: ++ case S_IFIFO: ++ if ((retval = do_mknod_internal(parent_ino, name, &st))) { ++ com_err(func_name, retval, ++ _("while creating special file \"%s\""), name); ++ return retval; ++ } ++ break; ++ case S_IFSOCK: ++ /* FIXME: there is no make sockect function atm. */ ++ com_err(func_name, 0, ++ _("ignoring sockect file\"%s\""), name); ++ break; ++ case S_IFLNK: ++ if((read_cnt = readlink(name, ln_target, sizeof(ln_target))) == -1) { ++ com_err(func_name, errno, ++ _("while trying to readlink \"%s\""), name); ++ return errno; ++ } ++ ln_target[read_cnt] = '\0'; ++ if ((retval = do_symlink_internal(parent_ino, name, ln_target))) { ++ com_err(func_name, retval, ++ _("while writing symlink\"%s\""), name); ++ return retval; ++ } ++ break; ++ case S_IFREG: ++ if ((retval = do_write_internal(parent_ino, name, name))) { ++ com_err(func_name, retval, ++ _("while writing file \"%s\""), name); ++ return retval; ++ } ++ break; ++ case S_IFDIR: ++ if ((retval = do_mkdir_internal(parent_ino, name, &st))) { ++ com_err(func_name, retval, ++ _("while making dir \"%s\""), name); ++ return retval; ++ } ++ if ((retval = ext2fs_namei(current_fs, root, parent_ino, name, &ino))) { ++ com_err(name, retval, 0); ++ return retval; ++ } ++ /* Populate the dir recursively*/ ++ retval = populate_fs(ino, name); ++ if (retval) { ++ com_err(func_name, retval, _("while adding dir \"%s\""), name); ++ return retval; ++ } ++ chdir(".."); ++ break; ++ default: ++ com_err(func_name, 0, ++ _("ignoring entry \"%s\""), name); ++ } ++ } ++ closedir(dh); ++ return retval; + } +-- +1.7.10.4 +
Signature · 1 line
1.8.1.2.545.g2f19ada
RFC PATCH 3/12 e2fsprogs/mke2fs: create special file
In reply to Mark HatleFrom: Robert Yang <[email protected]>
The do_mknod_internal() is used for creating special file which is block, character and fifo, most of the code are from debugfs/debugfs.c, the debugfs/debugfs.c will be modified to use this function.
--- .../0003-misc-util.c-create-special-file.patch | 98 ++++++++++++++++++++++ 1 file changed, 98 insertions(+) create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0003-misc-util.c-create-special-file.patch
meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0003-misc-util.c-create-special-file.patch new file +98 -0
@@ -0,0 +1,98 @@ +From de4bd6db5044263b15d4cbbbd0227fd2070b37da Mon Sep 17 00:00:00 2001 +From: Robert Yang <[email protected]> +Date: Tue, 27 Aug 2013 16:34:09 +0800 +Subject: [rootdir 03/10] misc/util.c: create special file + +The do_mknod_internal() is used for creating special file which is +block, character and fifo, most of the code are from debugfs/debugfs.c, +the debugfs/debugfs.c will be modified to use this function. + +Signed-off-by: Robert Yang <[email protected]> +--- + misc/util.c | 70 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 70 insertions(+) + +diff --git a/misc/util.c b/misc/util.c +index 6d87393..c793b42 100644 +--- a/misc/util.c ++++ b/misc/util.c +@@ -317,6 +317,76 @@ void dump_mmp_msg(struct mmp_struct *mmp, const char *msg) + /* Make a special file which is block, character and fifo */ + errcode_t do_mknod_internal(ext2_ino_t cwd, const char *name, struct stat *st) + { ++ ext2_ino_t ino; ++ errcode_t retval; ++ struct ext2_inode inode; ++ char *func_name = "do_mknod_internal"; ++ unsigned long major, minor, mode; ++ int filetype; ++ ++ switch(st->st_mode & S_IFMT) { ++ case S_IFCHR: ++ mode = LINUX_S_IFCHR; ++ filetype = EXT2_FT_CHRDEV; ++ break; ++ case S_IFBLK: ++ mode = LINUX_S_IFBLK; ++ filetype = EXT2_FT_BLKDEV; ++ break; ++ case S_IFIFO: ++ mode = LINUX_S_IFIFO; ++ filetype = EXT2_FT_FIFO; ++ break; ++ } ++ ++ if (!(current_fs->flags & EXT2_FLAG_RW)) { ++ com_err(func_name, 0, "Filesystem opened read/only"); ++ return -1; ++ } ++ retval = ext2fs_new_inode(current_fs, cwd, 010755, 0, &ino); ++ if (retval) { ++ com_err(func_name, retval, 0); ++ return retval; ++ } ++ printf("Allocated inode: %u\n", ino); ++ retval = ext2fs_link(current_fs, cwd, name, ino, filetype); ++ if (retval == EXT2_ET_DIR_NO_SPACE) { ++ retval = ext2fs_expand_dir(current_fs, cwd); ++ if (retval) { ++ com_err(func_name, retval, "while expanding directory"); ++ return retval; ++ } ++ retval = ext2fs_link(current_fs, cwd, name, ino, filetype); ++ } ++ if (retval) { ++ com_err(name, retval, 0); ++ return -1; ++ } ++ if (ext2fs_test_inode_bitmap2(current_fs->inode_map, ino)) ++ com_err(func_name, 0, "Warning: inode already set"); ++ ext2fs_inode_alloc_stats2(current_fs, ino, +1, 0); ++ memset(&inode, 0, sizeof(inode)); ++ inode.i_mode = mode; ++ inode.i_atime = inode.i_ctime = inode.i_mtime = ++ current_fs->now ? current_fs->now : time(0); ++ ++ major = major(st->st_rdev); ++ minor = minor(st->st_rdev); ++ ++ if ((major < 256) && (minor < 256)) { ++ inode.i_block[0] = major * 256 + minor; ++ inode.i_block[1] = 0; ++ } else { ++ inode.i_block[0] = 0; ++ inode.i_block[1] = (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12); ++ } ++ inode.i_links_count = 1; ++ ++ retval = ext2fs_write_new_inode(current_fs, ino, &inode); ++ if (retval) ++ com_err(func_name, retval, "while creating inode %u", ino); ++ ++ return retval; + } + + /* Make a symlink name -> target */ +-- +1.7.10.4 +
Signature · 1 line
1.8.1.2.545.g2f19ada
RFC PATCH 4/12 e2fsprogs/mke2fs: create symlink
In reply to Mark HatleFrom: Robert Yang <[email protected]>
The do_symlink_internal() is used for creating symlinks, most of the code are from debugfs/debugfs.c, the debugfs/debugfs.c will be modified to use this function.
--- .../0004-misc-util.c-create-symlink.patch | 62 ++++++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0004-misc-util.c-create-symlink.patch
meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0004-misc-util.c-create-symlink.patch new file +62 -0
@@ -0,0 +1,62 @@ +From f34058c69d2966b3e22dfe30b8f70533c3edbea6 Mon Sep 17 00:00:00 2001 +From: Robert Yang <[email protected]> +Date: Wed, 28 Aug 2013 09:58:13 +0800 +Subject: [rootdir 04/10] misc/util.c: create symlink + +The do_symlink_internal() is used for creating symlinks, most of the +code are from debugfs/debugfs.c, the debugfs/debugfs.c will be modified +to use this function. + +Signed-off-by: Robert Yang <[email protected]> +--- + misc/util.c | 34 ++++++++++++++++++++++++++++++++++ + 1 file changed, 34 insertions(+) + +diff --git a/misc/util.c b/misc/util.c +index c793b42..f6ae7ca 100644 +--- a/misc/util.c ++++ b/misc/util.c +@@ -392,6 +392,40 @@ errcode_t do_mknod_internal(ext2_ino_t cwd, const char *name, struct stat *st) + /* Make a symlink name -> target */ + errcode_t do_symlink_internal(ext2_ino_t cwd, const char *name, char *target) + { ++ char *cp; ++ ext2_ino_t parent_ino; ++ errcode_t retval; ++ struct ext2_inode inode; ++ struct stat st; ++ ++ cp = strrchr(name, '/'); ++ if (cp) { ++ *cp = 0; ++ if ((retval = ext2fs_namei(current_fs, root, cwd, name, &parent_ino))){ ++ com_err(name, retval, 0); ++ return retval; ++ } ++ name = cp+1; ++ } else { ++ parent_ino = cwd; ++ name = name; ++ } ++ ++try_again: ++ retval = ext2fs_symlink(current_fs, parent_ino, 0, name, target); ++ if (retval == EXT2_ET_DIR_NO_SPACE) { ++ retval = ext2fs_expand_dir(current_fs, parent_ino); ++ if (retval) { ++ com_err("do_symlink_internal", retval, "while expanding directory"); ++ return retval; ++ } ++ goto try_again; ++ } ++ if (retval) { ++ com_err("ext2fs_symlink", retval, 0); ++ return retval; ++ } ++ + } + + /* Make a directory in the fs */ +-- +1.7.10.4 +
Signature · 1 line
1.8.1.2.545.g2f19ada
RFC PATCH 5/12 e2fsprogs/mke2fs: copy regular file
In reply to Mark HatleFrom: Robert Yang <[email protected]>
The do_write_internal() is used for copying file from native fs to target, most of the code are from debugfs/debugfs.c, the debugfs/debugfs.c will be modified to use this function.
--- .../0005-misc-util.c-copy-regular-file.patch | 221 +++++++++++++++++++++ 1 file changed, 221 insertions(+) create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0005-misc-util.c-copy-regular-file.patch
meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0005-misc-util.c-copy-regular-file.patch new file +221 -0
@@ -0,0 +1,221 @@ +From 7c51a671c1de6237baaac4cf44bc7151394e521e Mon Sep 17 00:00:00 2001 +From: Robert Yang <[email protected]> +Date: Wed, 28 Aug 2013 10:06:11 +0800 +Subject: [rootdir 05/10] misc/util.c: copy regular file + +The do_write_internal() is used for copying file from native fs to +target, most of the code are from debugfs/debugfs.c, the +debugfs/debugfs.c will be modified to use this function. + +Signed-off-by: Robert Yang <[email protected]> +--- + misc/util.c | 179 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 179 insertions(+) + +diff --git a/misc/util.c b/misc/util.c +index f6ae7ca..6cb68ff 100644 +--- a/misc/util.c ++++ b/misc/util.c +@@ -37,6 +37,16 @@ + + #include "util.h" + ++/* 64KiB is the minimium blksize to best minimize system call overhead. */ ++#ifndef IO_BUFSIZE ++#define IO_BUFSIZE 64*1024 ++#endif ++ ++/* Block size for `st_blocks' */ ++#ifndef S_BLKSIZE ++#define S_BLKSIZE 512 ++#endif ++ + int journal_size; + int journal_flags; + char *journal_device; +@@ -428,6 +438,80 @@ try_again: + + } + ++static errcode_t copy_file(int fd, ext2_ino_t ino, int bufsize, int make_holes) ++{ ++ ext2_file_t e2_file; ++ errcode_t retval; ++ int got; ++ unsigned int written; ++ char *buf; ++ char *ptr; ++ char *zero_buf; ++ int cmp; ++ ++ retval = ext2fs_file_open(current_fs, ino, EXT2_FILE_WRITE, &e2_file); ++ if (retval) ++ return retval; ++ ++ if (!(buf = (char *) malloc(bufsize))){ ++ com_err("copy_file", errno, "can't allocate buffer\n"); ++ return; ++ } ++ ++ /* This is used for checking whether the whole block is zero */ ++ retval = ext2fs_get_memzero(bufsize, &zero_buf); ++ if (retval) { ++ com_err("copy_file", retval, "can't allocate buffer\n"); ++ free(buf); ++ return retval; ++ } ++ ++ while (1) { ++ got = read(fd, buf, bufsize); ++ if (got == 0) ++ break; ++ if (got < 0) { ++ retval = errno; ++ goto fail; ++ } ++ ptr = buf; ++ ++ /* Sparse copy */ ++ if (make_holes) { ++ /* Check whether all is zero */ ++ cmp = memcmp(ptr, zero_buf, got); ++ if (cmp == 0) { ++ /* The whole block is zero, make a hole */ ++ retval = ext2fs_file_lseek(e2_file, got, EXT2_SEEK_CUR, NULL); ++ if (retval) ++ goto fail; ++ got = 0; ++ } ++ } ++ ++ /* Normal copy */ ++ while (got > 0) { ++ retval = ext2fs_file_write(e2_file, ptr, ++ got, &written); ++ if (retval) ++ goto fail; ++ ++ got -= written; ++ ptr += written; ++ } ++ } ++ free(buf); ++ ext2fs_free_mem(&zero_buf); ++ retval = ext2fs_file_close(e2_file); ++ return retval; ++ ++fail: ++ free(buf); ++ ext2fs_free_mem(&zero_buf); ++ (void) ext2fs_file_close(e2_file); ++ return retval; ++} ++ + /* Make a directory in the fs */ + errcode_t do_mkdir_internal(ext2_ino_t cwd, const char *name, struct stat *st) + { +@@ -436,6 +520,101 @@ errcode_t do_mkdir_internal(ext2_ino_t cwd, const char *name, struct stat *st) + /* Copy the native file to the fs */ + errcode_t do_write_internal(ext2_ino_t cwd, const char *src, const char *dest) + { ++ int fd; ++ struct stat statbuf; ++ ext2_ino_t ino; ++ errcode_t retval; ++ struct ext2_inode inode; ++ char *func_name = "do_write_internal"; ++ int hdlink; ++ int bufsize = IO_BUFSIZE; ++ int make_holes = 0; ++ ++ fd = open(src, O_RDONLY); ++ if (fd < 0) { ++ com_err(src, errno, 0); ++ return errno; ++ } ++ if (fstat(fd, &statbuf) < 0) { ++ com_err(src, errno, 0); ++ close(fd); ++ return errno; ++ } ++ ++ retval = ext2fs_namei(current_fs, root, cwd, dest, &ino); ++ if (retval == 0) { ++ com_err(func_name, 0, "The file '%s' already exists\n", dest); ++ close(fd); ++ return errno; ++ } ++ ++ retval = ext2fs_new_inode(current_fs, cwd, 010755, 0, &ino); ++ if (retval) { ++ com_err(func_name, retval, 0); ++ close(fd); ++ return errno; ++ } ++ printf("Allocated inode: %u\n", ino); ++ retval = ext2fs_link(current_fs, cwd, dest, ino, EXT2_FT_REG_FILE); ++ if (retval == EXT2_ET_DIR_NO_SPACE) { ++ retval = ext2fs_expand_dir(current_fs, cwd); ++ if (retval) { ++ com_err(func_name, retval, "while expanding directory"); ++ close(fd); ++ return errno; ++ } ++ retval = ext2fs_link(current_fs, cwd, dest, ino, EXT2_FT_REG_FILE); ++ } ++ if (retval) { ++ com_err(dest, retval, 0); ++ close(fd); ++ return errno; ++ } ++ if (ext2fs_test_inode_bitmap2(current_fs->inode_map, ino)) ++ com_err(func_name, 0, "Warning: inode already set"); ++ ext2fs_inode_alloc_stats2(current_fs, ino, +1, 0); ++ memset(&inode, 0, sizeof(inode)); ++ inode.i_mode = (statbuf.st_mode & ~LINUX_S_IFMT) | LINUX_S_IFREG; ++ inode.i_atime = inode.i_ctime = inode.i_mtime = ++ current_fs->now ? current_fs->now : time(0); ++ inode.i_links_count = 1; ++ inode.i_size = statbuf.st_size; ++ if (current_fs->super->s_feature_incompat & ++ EXT3_FEATURE_INCOMPAT_EXTENTS) { ++ int i; ++ struct ext3_extent_header *eh; ++ ++ eh = (struct ext3_extent_header *) &inode.i_block[0]; ++ eh->eh_depth = 0; ++ eh->eh_entries = 0; ++ eh->eh_magic = EXT3_EXT_MAGIC; ++ i = (sizeof(inode.i_block) - sizeof(*eh)) / ++ sizeof(struct ext3_extent); ++ eh->eh_max = ext2fs_cpu_to_le16(i); ++ inode.i_flags |= EXT4_EXTENTS_FL; ++ } ++ ++ if ((retval = ext2fs_write_new_inode(current_fs, ino, &inode))) { ++ com_err(func_name, retval, "while creating inode %u", ino); ++ close(fd); ++ return errno; ++ } ++ if (LINUX_S_ISREG(inode.i_mode)) { ++ if (statbuf.st_blocks < statbuf.st_size / S_BLKSIZE) { ++ make_holes = 1; ++ /* ++ * Use I/O blocksize as buffer size when ++ * copying sparse files. ++ */ ++ bufsize = statbuf.st_blksize; ++ } ++ retval = copy_file(fd, ino, bufsize, make_holes); ++ if (retval) ++ com_err("copy_file", retval, 0); ++ } ++ close(fd); ++ ++ return 0; + } + + /* Copy files from source_dir to fs */ +-- +1.7.10.4 +
Signature · 1 line
1.8.1.2.545.g2f19ada
RFC PATCH 6/12 e2fsprogs/mke2fs: create directory
In reply to Mark HatleFrom: Robert Yang <[email protected]>
The do_mkdir_internal() is used for making dir on the target fs, most of the code are from debugfs/debugfs.c, the debugfs/debugfs.c will be modified to use this function.
--- .../0006-misc-util.c-create-directory.patch | 62 ++++++++++++++++++++++ 1 file changed, 62 insertions(+) create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0006-misc-util.c-create-directory.patch
meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0006-misc-util.c-create-directory.patch new file +62 -0
@@ -0,0 +1,62 @@ +From 41344adccc0bf38893ff1c402301357e8a174713 Mon Sep 17 00:00:00 2001 +From: Robert Yang <[email protected]> +Date: Wed, 28 Aug 2013 10:14:38 +0800 +Subject: [rootdir 06/10] misc/util.c: create directory + +The do_mkdir_internal() is used for making dir on the target fs, most of +the code are from debugfs/debugfs.c, the debugfs/debugfs.c will be +modified to use this function. + +Signed-off-by: Robert Yang <[email protected]> +--- + misc/util.c | 34 ++++++++++++++++++++++++++++++++++ + 1 file changed, 34 insertions(+) + +diff --git a/misc/util.c b/misc/util.c +index e35fd88..7a45120 100644 +--- a/misc/util.c ++++ b/misc/util.c +@@ -471,6 +471,40 @@ fail: + /* Make a directory in the fs */ + errcode_t do_mkdir_internal(ext2_ino_t cwd, const char *name, struct stat *st) + { ++ char *cp; ++ ext2_ino_t parent_ino, ino; ++ errcode_t retval; ++ struct ext2_inode inode; ++ char *func_name = "do_mkdir_internal"; ++ ++ ++ cp = strrchr(name, '/'); ++ if (cp) { ++ *cp = 0; ++ if ((retval = ext2fs_namei(current_fs, root, cwd, name, &parent_ino))){ ++ com_err(name, retval, 0); ++ return retval; ++ } ++ name = cp+1; ++ } else { ++ parent_ino = cwd; ++ name = name; ++ } ++ ++try_again: ++ retval = ext2fs_mkdir(current_fs, parent_ino, 0, name); ++ if (retval == EXT2_ET_DIR_NO_SPACE) { ++ retval = ext2fs_expand_dir(current_fs, parent_ino); ++ if (retval) { ++ com_err(func_name, retval, "while expanding directory"); ++ return retval; ++ } ++ goto try_again; ++ } ++ if (retval) { ++ com_err("ext2fs_mkdir", retval, 0); ++ return retval; ++ } + } + + /* Copy the native file to the fs */ +-- +1.7.10.4 +
Signature · 1 line
1.8.1.2.545.g2f19ada
RFC PATCH 7/12 e2fsprogs/mke2fs: set more information for inode
In reply to Mark HatleFrom: Robert Yang <[email protected]>
Set the uid, gid, mode and time for inode.
--- ...isc-util.c-set-more-information-for-inode.patch | 79 ++++++++++++++++++++++ 1 file changed, 79 insertions(+) create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0007-misc-util.c-set-more-information-for-inode.patch
meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0007-misc-util.c-set-more-information-for-inode.patch new file +79 -0
@@ -0,0 +1,79 @@ +From 760eb73420f13d99244d1d23d69c890fd29d8fdc Mon Sep 17 00:00:00 2001 +From: Robert Yang <[email protected]> +Date: Wed, 28 Aug 2013 10:18:01 +0800 +Subject: [rootdir 07/10] misc/util.c: set more information for inode + +Set the uid, gid, mode and time for inode. + +Signed-off-by: Robert Yang <[email protected]> +--- + misc/util.c | 46 ++++++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 46 insertions(+) + +diff --git a/misc/util.c b/misc/util.c +index 7a45120..1a0a58c 100644 +--- a/misc/util.c ++++ b/misc/util.c +@@ -314,6 +314,41 @@ void dump_mmp_msg(struct mmp_struct *mmp, const char *msg) + } + } + ++/* Fill the uid, gid, mode and time for the inode */ ++static void fill_inode(struct ext2_inode *inode, struct stat *st) ++{ ++ if (st != NULL) { ++ inode->i_uid = st->st_uid; ++ inode->i_gid = st->st_gid; ++ inode->i_mode |= st->st_mode; ++ inode->i_atime = st->st_atime; ++ inode->i_mtime = st->st_mtime; ++ inode->i_ctime = st->st_ctime; ++ } ++} ++ ++/* Set the uid, gid, mode and time for the inode */ ++errcode_t set_inode_extra(ext2_ino_t cwd, ext2_ino_t ino, struct stat *st) ++{ ++ errcode_t retval; ++ struct ext2_inode inode; ++ char *func_name = "set_inode_extra"; ++ ++ retval = ext2fs_read_inode(current_fs, ino, &inode); ++ if (retval) { ++ com_err(func_name, retval, "while reading inode %u", ino); ++ return retval; ++ } ++ ++ fill_inode(&inode, st); ++ ++ retval = ext2fs_write_inode(current_fs, ino, &inode); ++ if (retval) { ++ com_err(func_name, retval, "while writing inode %u", ino); ++ return retval; ++ } ++} ++ + /* Make a special file which is block, character and fifo */ + errcode_t do_mknod_internal(ext2_ino_t cwd, const char *name, struct stat *st) + { +@@ -687,6 +722,17 @@ errcode_t populate_fs(ext2_ino_t parent_ino, const char *source_dir) + com_err(func_name, 0, + _("ignoring entry \"%s\""), name); + } ++ ++ if ((retval = ext2fs_namei(current_fs, root, parent_ino, name, &ino))){ ++ com_err(name, retval, 0); ++ return retval; ++ } ++ ++ if ((retval = set_inode_extra(parent_ino, ino, &st))) { ++ com_err(func_name, retval, ++ _("while setting inode for \"%s\""), name); ++ return retval; ++ } + } + closedir(dh); + return retval; +-- +1.7.10.4 +
Signature · 1 line
1.8.1.2.545.g2f19ada
RFC PATCH 8/12 e2fsprogs/mke2fs: handle hardlinks
In reply to Mark HatleFrom: Robert Yang <[email protected]>
Create the file and save the source inode number when we meet the hard link at the first time, use ext2fs_link() to link the name to the target inode number when we meet the same source inode number again.
--- .../0008-misc-util.c-handle-hardlinks.patch | 145 +++++++++++++++++++++ 1 file changed, 145 insertions(+) create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0008-misc-util.c-handle-hardlinks.patch
meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0008-misc-util.c-handle-hardlinks.patch new file +145 -0
@@ -0,0 +1,145 @@ +From 9d9c23a0614c33891ec459afdfeb201425c3c4bb Mon Sep 17 00:00:00 2001 +From: Robert Yang <[email protected]> +Date: Wed, 28 Aug 2013 10:27:05 +0800 +Subject: [rootdir 08/10] misc/util.c: handle hardlinks + +Create the file and save the source inode number when we meet the hard +link at the first time, use ext2fs_link() to link the name to the target +inode number when we meet the same source inode number again. + +Signed-off-by: Robert Yang <[email protected]> +--- + misc/util.c | 85 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ + 1 file changed, 85 insertions(+) + +diff --git a/misc/util.c b/misc/util.c +index 1a0a58c..11134db 100644 +--- a/misc/util.c ++++ b/misc/util.c +@@ -314,6 +314,42 @@ void dump_mmp_msg(struct mmp_struct *mmp, const char *msg) + } + } + ++/* Link an inode number to a directory */ ++static errcode_t add_link(ext2_ino_t parent_ino, ext2_ino_t ino, const char *name) ++{ ++ struct ext2_inode inode; ++ errcode_t retval; ++ char *func_name = "add_to_dir"; ++ ++ retval = ext2fs_read_inode(current_fs, ino, &inode); ++ if (retval) { ++ com_err(func_name, retval, "while reading inode %u", ino); ++ return retval; ++ } ++ ++ retval = ext2fs_link(current_fs, parent_ino, name, ino, inode.i_flags); ++ if (retval == EXT2_ET_DIR_NO_SPACE) { ++ retval = ext2fs_expand_dir(current_fs, parent_ino); ++ if (retval) { ++ com_err(func_name, retval, "while expanding directory"); ++ return retval; ++ } ++ retval = ext2fs_link(current_fs, parent_ino, name, ino, inode.i_flags); ++ } ++ if (retval) { ++ com_err(func_name, retval, "while linking %s", name); ++ return retval; ++ } ++ ++ inode.i_links_count++; ++ ++ retval = ext2fs_write_inode(current_fs, ino, &inode); ++ if (retval) ++ com_err(func_name, retval, "while writing inode %u", ino); ++ ++ return retval; ++} ++ + /* Fill the uid, gid, mode and time for the inode */ + static void fill_inode(struct ext2_inode *inode, struct stat *st) + { +@@ -542,6 +578,17 @@ try_again: + } + } + ++int is_hardlink(ext2_ino_t ino) ++{ ++ int i; ++ ++ for(i = 0; i < hdlinks.count; i++) { ++ if(hdlinks.hdl[i].src_ino == ino) ++ return i; ++ } ++ return -1; ++} ++ + /* Copy the native file to the fs */ + errcode_t do_write_internal(ext2_ino_t cwd, const char *src, const char *dest) + { +@@ -640,10 +687,12 @@ errcode_t populate_fs(ext2_ino_t parent_ino, const char *source_dir) + struct dirent *dent; + struct stat st; + char ln_target[PATH_MAX]; ++ unsigned int save_inode = 0; + char *func_name = "populate_fs"; + ext2_ino_t ino; + errcode_t retval; + int read_cnt; ++ int hdlink; + + root = EXT2_ROOT_INO; + +@@ -665,6 +714,21 @@ errcode_t populate_fs(ext2_ino_t parent_ino, const char *source_dir) + lstat(dent->d_name, &st); + name = dent->d_name; + ++ /* Check for hardlinks */ ++ if (!S_ISDIR(st.st_mode) && !S_ISLNK(st.st_mode) && st.st_nlink > 1) { ++ hdlink = is_hardlink(st.st_ino); ++ if (hdlink >= 0) { ++ retval = add_link(parent_ino, ++ hdlinks.hdl[hdlink].dst_ino, name); ++ if (retval) { ++ com_err(func_name, retval, "while linking %s", name); ++ return retval; ++ } ++ continue; ++ } else ++ save_inode = 1; ++ } ++ + switch(st.st_mode & S_IFMT) { + case S_IFCHR: + case S_IFBLK: +@@ -733,6 +797,27 @@ errcode_t populate_fs(ext2_ino_t parent_ino, const char *source_dir) + _("while setting inode for \"%s\""), name); + return retval; + } ++ ++ /* Save the hardlink ino */ ++ if (save_inode) { ++ /* ++ * Check whether need more memory, and we don't need ++ * free() since the lifespan will be over after the fs ++ * populated. ++ */ ++ if (hdlinks.count == hdlink_cnt) { ++ if ((hdlinks.hdl = realloc (hdlinks.hdl, ++ (hdlink_cnt + HDLINK_CNT) * ++ sizeof (struct hdlink_s))) == NULL) { ++ com_err(name, errno, "Not enough memory"); ++ return errno; ++ } ++ hdlink_cnt += HDLINK_CNT; ++ } ++ hdlinks.hdl[hdlinks.count].src_ino = st.st_ino; ++ hdlinks.hdl[hdlinks.count].dst_ino = ino; ++ hdlinks.count++; ++ } + } + closedir(dh); + return retval; +-- +1.7.10.4 +
Signature · 1 line
1.8.1.2.545.g2f19ada
RFC PATCH 9/12 e2fsprogs/mke2fs: update the manual for the -d option
In reply to Mark HatleFrom: Robert Yang <[email protected]>
update the manual for the -d option
--- ...s.8.in-update-the-manual-for-the-d-option.patch | 40 ++++++++++++++++++++++ 1 file changed, 40 insertions(+) create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0009-mke2fs.8.in-update-the-manual-for-the-d-option.patch
meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0009-mke2fs.8.in-update-the-manual-for-the-d-option.patch new file +40 -0
@@ -0,0 +1,40 @@ +From 1f8d6c42c4c3ac3494f603ef57c197f35a220bd5 Mon Sep 17 00:00:00 2001 +From: Robert Yang <[email protected]> +Date: Tue, 27 Aug 2013 15:55:35 +0800 +Subject: [rootdir 09/10] mke2fs.8.in: update the manual for the -d option + +update the manual for the -d option + +Signed-off-by: Robert Yang <[email protected]> +--- + misc/mke2fs.8.in | 7 +++++++ + 1 file changed, 7 insertions(+) + +diff --git a/misc/mke2fs.8.in b/misc/mke2fs.8.in +index fe2dcdb..b268154 100644 +--- a/misc/mke2fs.8.in ++++ b/misc/mke2fs.8.in +@@ -52,6 +52,10 @@ mke2fs \- create an ext2/ext3/ext4 filesystem + .I number-of-inodes + ] + [ ++.B \-d ++.I root-directory ++] ++[ + .B \-n + ] + [ +@@ -490,6 +494,9 @@ the + ratio). This allows the user to specify the number + of desired inodes directly. + .TP ++.BI \-d " root-directory" ++Add the files from the root-directory to the filesystem. ++.TP + .BI \-o " creator-os" + Overrides the default value of the "creator operating system" field of the + filesystem. The creator field is set by default to the name of the OS the +-- +1.7.10.4 +
Signature · 1 line
1.8.1.2.545.g2f19ada
RFC PATCH 10/12 e2fsprogs/mke2fs: debugfs: use the functions in misc/util.c
In reply to Mark HatleFrom: Robert Yang <[email protected]>
Use the functions in misc/util.c to reduce the duplicated code.
We should use the $(top_builddir)/misc/util.o rather than $(srcdir)/../misc/util.o, otherwise, if B != S, then the util.o would be placed in $(srcdir) which would cause build errors.
--- ...-debugfs-use-the-functions-in-misc-util.c.patch | 449 +++++++++++++++++++++ 1 file changed, 449 insertions(+) create mode 100644 meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0010-debugfs-use-the-functions-in-misc-util.c.patch
meta/recipes-devtools/e2fsprogs/e2fsprogs-1.42.8/0010-debugfs-use-the-functions-in-misc-util.c.patch new file +449 -0
@@ -0,0 +1,449 @@ +From 08443658349aaf3bfa7666ff9445c9170ef6784f Mon Sep 17 00:00:00 2001 +From: Robert Yang <[email protected]> +Date: Wed, 28 Aug 2013 10:32:59 +0800 +Subject: [rootdir 10/10] debugfs: use the functions in misc/util.c + +Use the functions in misc/util.c to reduce the duplicated code. + +Signed-off-by: Robert Yang <[email protected]> +--- + debugfs/Makefile.in | 7 +- + debugfs/debugfs.c | 294 +++------------------------------------------------ + debugfs/debugfs.h | 1 + + 3 files changed, 21 insertions(+), 281 deletions(-) + +diff --git a/debugfs/Makefile.in b/debugfs/Makefile.in +index 9a86dc6..4af2483 100644 +--- a/debugfs/Makefile.in ++++ b/debugfs/Makefile.in +@@ -18,7 +18,7 @@ MK_CMDS= _SS_DIR_OVERRIDE=../lib/ss ../lib/ss/mk_cmds + + DEBUG_OBJS= debug_cmds.o debugfs.o util.o ncheck.o icheck.o ls.o \ + lsdel.o dump.o set_fields.o logdump.o htree.o unused.o e2freefrag.o \ +- filefrag.o extent_cmds.o extent_inode.o zap.o ++ filefrag.o extent_cmds.o extent_inode.o zap.o $(top_builddir)/misc/util.o + + RO_DEBUG_OBJS= ro_debug_cmds.o ro_debugfs.o util.o ncheck.o icheck.o ls.o \ + lsdel.o logdump.o htree.o e2freefrag.o filefrag.o extent_cmds.o \ +@@ -28,7 +28,8 @@ SRCS= debug_cmds.c $(srcdir)/debugfs.c $(srcdir)/util.c $(srcdir)/ls.c \ + $(srcdir)/ncheck.c $(srcdir)/icheck.c $(srcdir)/lsdel.c \ + $(srcdir)/dump.c $(srcdir)/set_fields.c ${srcdir}/logdump.c \ + $(srcdir)/htree.c $(srcdir)/unused.c ${srcdir}/../misc/e2freefrag.c \ +- $(srcdir)/filefrag.c $(srcdir)/extent_inode.c $(srcdir)/zap.c ++ $(srcdir)/filefrag.c $(srcdir)/extent_inode.c $(srcdir)/zap.c \ ++ $(srcdir)/../misc/util.c + + LIBS= $(LIBEXT2FS) $(LIBE2P) $(LIBSS) $(LIBCOM_ERR) $(LIBBLKID) \ + $(LIBUUID) +@@ -141,7 +142,7 @@ debugfs.o: $(srcdir)/debugfs.c $(top_srcdir)/lib/et/com_err.h \ + $(top_srcdir)/lib/ext2fs/ext2_ext_attr.h $(top_srcdir)/lib/ext2fs/bitops.h \ + $(top_srcdir)/lib/e2p/e2p.h $(top_srcdir)/version.h $(srcdir)/jfs_user.h \ + $(top_srcdir)/lib/ext2fs/kernel-jbd.h $(top_srcdir)/lib/ext2fs/jfs_compat.h \ +- $(top_srcdir)/lib/ext2fs/kernel-list.h ++ $(top_srcdir)/lib/ext2fs/kernel-list.h $(top_srcdir)/misc/util.h + util.o: $(srcdir)/util.c $(srcdir)/debugfs.h \ + $(top_srcdir)/lib/ext2fs/ext2_fs.h $(top_builddir)/lib/ext2fs/ext2_types.h \ + $(top_srcdir)/lib/ext2fs/ext2fs.h $(top_srcdir)/lib/ext2fs/ext3_extents.h \ +diff --git a/debugfs/debugfs.c b/debugfs/debugfs.c +index 5590295..5d2b289 100644 +--- a/debugfs/debugfs.c ++++ b/debugfs/debugfs.c +@@ -25,8 +25,6 @@ extern char *optarg; + #include <errno.h> + #endif + #include <fcntl.h> +-#include <sys/types.h> +-#include <sys/stat.h> + + #include "debugfs.h" + #include "uuid/uuid.h" +@@ -55,8 +53,7 @@ ss_request_table *extra_cmds; + const char *debug_prog_name; + int sci_idx; + +-ext2_filsys current_fs = NULL; +-ext2_ino_t root, cwd; ++ext2_ino_t cwd; + + static void open_filesystem(char *device, int open_flags, blk64_t superblock, + blk64_t blocksize, int catastrophic, +@@ -1573,188 +1570,24 @@ void do_find_free_inode(int argc, char *argv[]) + } + + #ifndef READ_ONLY +-static errcode_t copy_file(int fd, ext2_ino_t newfile, int bufsize, int make_holes) +-{ +- ext2_file_t e2_file; +- errcode_t retval; +- int got; +- unsigned int written; +- char *buf; +- char *ptr; +- char *zero_buf; +- int cmp; +- +- retval = ext2fs_file_open(current_fs, newfile, +- EXT2_FILE_WRITE, &e2_file); +- if (retval) +- return retval; +- +- if (!(buf = (char *) malloc(bufsize))){ +- com_err("copy_file", errno, "can't allocate buffer\n"); +- return; +- } +- +- /* This is used for checking whether the whole block is zero */ +- retval = ext2fs_get_memzero(bufsize, &zero_buf); +- if (retval) { +- com_err("copy_file", retval, "can't allocate buffer\n"); +- free(buf); +- return retval; +- } +- +- while (1) { +- got = read(fd, buf, bufsize); +- if (got == 0) +- break; +- if (got < 0) { +- retval = errno; +- goto fail; +- } +- ptr = buf; +- +- /* Sparse copy */ +- if (make_holes) { +- /* Check whether all is zero */ +- cmp = memcmp(ptr, zero_buf, got); +- if (cmp == 0) { +- /* The whole block is zero, make a hole */ +- retval = ext2fs_file_lseek(e2_file, got, EXT2_SEEK_CUR, NULL); +- if (retval) +- goto fail; +- got = 0; +- } +- } +- +- /* Normal copy */ +- while (got > 0) { +- retval = ext2fs_file_write(e2_file, ptr, +- got, &written); +- if (retval) +- goto fail; +- +- got -= written; +- ptr += written; +- } +- } +- free(buf); +- ext2fs_free_mem(&zero_buf); +- retval = ext2fs_file_close(e2_file); +- return retval; +- +-fail: +- free(buf); +- ext2fs_free_mem(&zero_buf); +- (void) ext2fs_file_close(e2_file); +- return retval; +-} +- +- + void do_write(int argc, char *argv[]) + { +- int fd; +- struct stat statbuf; +- ext2_ino_t newfile; + errcode_t retval; +- struct ext2_inode inode; +- int bufsize = IO_BUFSIZE; +- int make_holes = 0; + + if (common_args_process(argc, argv, 3, 3, "write", + "<native file> <new file>", CHECK_FS_RW)) + return; + +- fd = open(argv[1], O_RDONLY); +- if (fd < 0) { +- com_err(argv[1], errno, 0); +- return; +- } +- if (fstat(fd, &statbuf) < 0) { +- com_err(argv[1], errno, 0); +- close(fd); +- return; +- } +- +- retval = ext2fs_namei(current_fs, root, cwd, argv[2], &newfile); +- if (retval == 0) { +- com_err(argv[0], 0, "The file '%s' already exists\n", argv[2]); +- close(fd); +- return; +- } +- +- retval = ext2fs_new_inode(current_fs, cwd, 010755, 0, &newfile); +- if (retval) { ++ if ((retval = do_write_internal(cwd, argv[1], argv[2]))) + com_err(argv[0], retval, 0); +- close(fd); +- return; +- } +- printf("Allocated inode: %u\n", newfile); +- retval = ext2fs_link(current_fs, cwd, argv[2], newfile, +- EXT2_FT_REG_FILE); +- if (retval == EXT2_ET_DIR_NO_SPACE) { +- retval = ext2fs_expand_dir(current_fs, cwd); +- if (retval) { +- com_err(argv[0], retval, "while expanding directory"); +- close(fd); +- return; +- } +- retval = ext2fs_link(current_fs, cwd, argv[2], newfile, +- EXT2_FT_REG_FILE); +- } +- if (retval) { +- com_err(argv[2], retval, 0); +- close(fd); +- return; +- } +- if (ext2fs_test_inode_bitmap2(current_fs->inode_map,newfile)) +- com_err(argv[0], 0, "Warning: inode already set"); +- ext2fs_inode_alloc_stats2(current_fs, newfile, +1, 0); +- memset(&inode, 0, sizeof(inode)); +- inode.i_mode = (statbuf.st_mode & ~LINUX_S_IFMT) | LINUX_S_IFREG; +- inode.i_atime = inode.i_ctime = inode.i_mtime = +- current_fs->now ? current_fs->now : time(0); +- inode.i_links_count = 1; +- inode.i_size = statbuf.st_size; +- if (current_fs->super->s_feature_incompat & +- EXT3_FEATURE_INCOMPAT_EXTENTS) { +- int i; +- struct ext3_extent_header *eh; +- +- eh = (struct ext3_extent_header *) &inode.i_block[0]; +- eh->eh_depth = 0; +- eh->eh_entries = 0; +- eh->eh_magic = EXT3_EXT_MAGIC; +- i = (sizeof(inode.i_block) - sizeof(*eh)) / +- sizeof(struct ext3_extent); +- eh->eh_max = ext2fs_cpu_to_le16(i); +- inode.i_flags |= EXT4_EXTENTS_FL; +- } +- if (debugfs_write_new_inode(newfile, &inode, argv[0])) { +- close(fd); +- return; +- } +- if (LINUX_S_ISREG(inode.i_mode)) { +- if (statbuf.st_blocks < statbuf.st_size / S_BLKSIZE) { +- make_holes = 1; +- /* +- * Use I/O blocksize as buffer size when +- * copying sparse files. +- */ +- bufsize = statbuf.st_blksize; +- } +- retval = copy_file(fd, newfile, bufsize, make_holes); +- if (retval) +- com_err("copy_file", retval, 0); +- } +- close(fd); + } + + void do_mknod(int argc, char *argv[]) + { + unsigned long mode, major, minor; +- ext2_ino_t newfile; + errcode_t retval; +- struct ext2_inode inode; + int filetype, nr; ++ struct stat st; + + if (check_fs_open(argv[0])) + return; +@@ -1763,115 +1596,50 @@ void do_mknod(int argc, char *argv[]) + com_err(argv[0], 0, "Usage: mknod <name> [p| [c|b] <major> <minor>]"); + return; + } ++ + mode = minor = major = 0; + switch (argv[2][0]) { + case 'p': +- mode = LINUX_S_IFIFO; +- filetype = EXT2_FT_FIFO; ++ st.st_mode = S_IFIFO; + nr = 3; + break; + case 'c': +- mode = LINUX_S_IFCHR; +- filetype = EXT2_FT_CHRDEV; ++ st.st_mode = S_IFCHR; + nr = 5; + break; + case 'b': +- mode = LINUX_S_IFBLK; +- filetype = EXT2_FT_BLKDEV; ++ st.st_mode = S_IFBLK; + nr = 5; + break; + default: +- filetype = 0; + nr = 0; + } ++ + if (nr == 5) { + major = strtoul(argv[3], argv+3, 0); + minor = strtoul(argv[4], argv+4, 0); + if (major > 65535 || minor > 65535 || argv[3][0] || argv[4][0]) + nr = 0; + } ++ + if (argc != nr) + goto usage; +- if (check_fs_read_write(argv[0])) +- return; +- retval = ext2fs_new_inode(current_fs, cwd, 010755, 0, &newfile); +- if (retval) { ++ ++ st.st_rdev = makedev(major, minor); ++ if ((retval = do_mknod_internal(cwd, argv[1], &st))) + com_err(argv[0], retval, 0); +- return; +- } +- printf("Allocated inode: %u\n", newfile); +- retval = ext2fs_link(current_fs, cwd, argv[1], newfile, filetype); +- if (retval == EXT2_ET_DIR_NO_SPACE) { +- retval = ext2fs_expand_dir(current_fs, cwd); +- if (retval) { +- com_err(argv[0], retval, "while expanding directory"); +- return; +- } +- retval = ext2fs_link(current_fs, cwd, argv[1], newfile, +- filetype); +- } +- if (retval) { +- com_err(argv[1], retval, 0); +- return; +- } +- if (ext2fs_test_inode_bitmap2(current_fs->inode_map,newfile)) +- com_err(argv[0], 0, "Warning: inode already set"); +- ext2fs_inode_alloc_stats2(current_fs, newfile, +1, 0); +- memset(&inode, 0, sizeof(inode)); +- inode.i_mode = mode; +- inode.i_atime = inode.i_ctime = inode.i_mtime = +- current_fs->now ? current_fs->now : time(0); +- if ((major < 256) && (minor < 256)) { +- inode.i_block[0] = major*256+minor; +- inode.i_block[1] = 0; +- } else { +- inode.i_block[0] = 0; +- inode.i_block[1] = (minor & 0xff) | (major << 8) | ((minor & ~0xff) << 12); +- } +- inode.i_links_count = 1; +- if (debugfs_write_new_inode(newfile, &inode, argv[0])) +- return; + } + + void do_mkdir(int argc, char *argv[]) + { +- char *cp; +- ext2_ino_t parent; +- char *name; + errcode_t retval; + + if (common_args_process(argc, argv, 2, 2, "mkdir", + "<filename>", CHECK_FS_RW)) + return; + +- cp = strrchr(argv[1], '/'); +- if (cp) { +- *cp = 0; +- parent = string_to_inode(argv[1]); +- if (!parent) { +- com_err(argv[1], ENOENT, 0); +- return; +- } +- name = cp+1; +- } else { +- parent = cwd; +- name = argv[1]; +- } +- +-try_again: +- retval = ext2fs_mkdir(current_fs, parent, 0, name); +- if (retval == EXT2_ET_DIR_NO_SPACE) { +- retval = ext2fs_expand_dir(current_fs, parent); +- if (retval) { +- com_err(argv[0], retval, "while expanding directory"); +- return; +- } +- goto try_again; +- } +- if (retval) { +- com_err("ext2fs_mkdir", retval, 0); +- return; +- } ++ if ((retval = do_mkdir_internal(cwd, argv[1], NULL))) ++ com_err(argv[0], retval, 0); + + } + +@@ -2260,44 +2028,14 @@ void do_punch(int argc, char *argv[]) + + void do_symlink(int argc, char *argv[]) + { +- char *cp; +- ext2_ino_t parent; +- char *name, *target; + errcode_t retval; + + if (common_args_process(argc, argv, 3, 3, "symlink", + "<filename> <target>", CHECK_FS_RW)) + return; + +- cp = strrchr(argv[1], '/'); +- if (cp) { +- *cp = 0; +- parent = string_to_inode(argv[1]); +- if (!parent) { +- com_err(argv[1], ENOENT, 0); +- return; +- } +- name = cp+1; +- } else { +- parent = cwd; +- name = argv[1]; +- } +- target = argv[2]; +- +-try_again: +- retval = ext2fs_symlink(current_fs, parent, 0, name, target); +- if (retval == EXT2_ET_DIR_NO_SPACE) { +- retval = ext2fs_expand_dir(current_fs, parent); +- if (retval) { +- com_err(argv[0], retval, "while expanding directory"); +- return; +- } +- goto try_again; +- } +- if (retval) { +- com_err("ext2fs_symlink", retval, 0); +- return; +- } ++ if ((retval = do_symlink_internal(cwd, argv[1], argv[2]))) ++ com_err(argv[0], retval, 0); + + } + +diff --git a/debugfs/debugfs.h b/debugfs/debugfs.h +index 45175cf..64ed608 100644 +--- a/debugfs/debugfs.h ++++ b/debugfs/debugfs.h +@@ -5,6 +5,7 @@ + #include "ss/ss.h" + #include "ext2fs/ext2_fs.h" + #include "ext2fs/ext2fs.h" ++#include "../misc/util.h" + + #ifdef __STDC__ + #define NOARGS void +-- +1.7.10.4 +
Signature · 1 line
1.8.1.2.545.g2f19ada
RFC PATCH 11/12 e2fsprogs: enable the "-d" related patches
In reply to Mark HatleFrom: Robert Yang <[email protected]>
These patches should be enabled at the same time, so enable them in a single commit, and thus we can disable it easily.
--- meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.8.bb | 10 ++++++++++ 1 file changed, 10 insertions(+)
meta/recipes-devtools/e2fsprogs/e2fsprogs_1.42.8.bb +10 -0
@@ -9,6 +9,16 @@ SRC_URI += "file://acinclude.m4 \ file://debugfs-extent-header.patch \ file://populate-extfs.sh \ file://e2fsprogs-fix-tests-f_extent_oobounds.patch \ + file://0001-mke2fs.c-add-an-option-d-root-directory.patch \ + file://0002-misc-util.c-implement-populate_fs.patch \ + file://0003-misc-util.c-create-special-file.patch \ + file://0004-misc-util.c-create-symlink.patch \ + file://0005-misc-util.c-copy-regular-file.patch \ + file://0006-misc-util.c-create-directory.patch \ + file://0007-misc-util.c-set-more-information-for-inode.patch \ + file://0008-misc-util.c-handle-hardlinks.patch \ + file://0009-mke2fs.8.in-update-the-manual-for-the-d-option.patch \ + file://0010-debugfs-use-the-functions-in-misc-util.c.patch \ " SRC_URI[md5sum] = "8ef664b6eb698aa6b733df59b17b9ed4"
Signature · 1 line
1.8.1.2.545.g2f19ada
RFC PATCH 12/12 image_types.bbclass: use mke2fs -d to create the ext2/3/4 image
In reply to Mark HatleFrom: Robert Yang <[email protected]>
We used populate-extfs.sh which invoked the debugfs to create the image, now the mke2fs' option "-d root-directory" can do the same thing, and which is a little faster, for example, the glibc_std:
$ time mkfs $ time populate-extfs.sh /path/to/rootfs/ wrlinux-image-glibc-std-qemux86.ext4 real 1m3.724s user 0m54.955s sys 0m8.021s
$ time mkfs.ext4 -F -d /path/to/rootfs/ wrlinux-image-glibc-std-qemux86.ext4 real 0m46.256s user 0m40.323s sys 0m1.628s
17s are saved.
--- meta/classes/image_types.bbclass | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
meta/classes/image_types.bbclass +1 -2
@@ -152,8 +152,7 @@ oe_mkext234fs () { # Create a sparse image block dd if=/dev/zero of=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype seek=$ROOTFS_SIZE count=0 bs=1k - mkfs.$fstype -F $extra_imagecmd ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype - populate-extfs.sh ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype + mkfs.$fstype -F $extra_imagecmd ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.$fstype -d ${IMAGE_ROOTFS} } IMAGE_CMD_ext2 = "oe_mkext234fs ext2 ${EXTRA_IMAGECMD}"
Signature · 1 line
1.8.1.2.545.g2f19ada