Skip to contentSkip to conversation

Initial extents support for e2fsprogs

RFC PATCH New Theodore Ts'o 1 participant 5 messages Sep 17, 2007

lore
Share & tools
lore
#0 lore raw

For people who are curious about how the extents patch for e2fsprogs is going, here's what I currently have as a work in progress. With this patch, e2fsck will currently correctly handle a valid filesystem that contains extents, but it's missing a lot of sanity checks. In particular, the pass1b code hasn't been enhanced to deal with extents yet, plus checks to make sure the extent tree are sane are missing. But it should be enough for people to get the general idea of where I'm going.

- Ted

#1 lore raw

RFC PATCH 1/4 Allow debugfs to be extended for use by test programs

This change allows debugfs to be reused as the base for e2fsprogs internal test programs, by linking debugfs object files with additional object file(s) that define additional commands. The test program's object file(s) should define their own comand table, and define the symbol extra_cmds to be a pointer to the ss_request_table. In addition, the symbol debug_prog_name can be used to override the name of the program printed in the version banner and in the ss prompt.

---
 debugfs/debugfs.c |   21 ++++++++++++++++-----
 1 files changed, 16 insertions(+), 5 deletions(-)
debugfs/debugfs.c +16 -5
index 190c4b7..76542d8 100644
@@ -38,6 +38,8 @@ extern char *optarg;
 #include "../version.h"
 
 extern ss_request_table debug_cmds;
+ss_request_table *extra_cmds;
+char *debug_prog_name;
 
 ext2_filsys	current_fs = NULL;
 ext2_ino_t	root, cwd;
@@ -1802,7 +1804,7 @@ int main(int argc, char **argv)
 {
 	int		retval;
 	int		sci_idx;
-	const char	*usage = "Usage: debugfs [-b blocksize] [-s superblock] [-f cmd_file] [-R request] [-V] [[-w] [-c] device]";
+	const char	*usage = "Usage: %s [-b blocksize] [-s superblock] [-f cmd_file] [-R request] [-V] [[-w] [-c] device]";
 	int		c;
 	int		open_flags = EXT2_FLAG_SOFTSUPP_FEATURES;
 	char		*request = 0;
@@ -1813,9 +1815,12 @@ int main(int argc, char **argv)
 	int		catastrophic = 0;
 	char		*data_filename = 0;
 	
+	if (debug_prog_name == 0)
+		debug_prog_name = "debugfs";
+
 	add_error_table(&et_ext2_error_table);
-	fprintf (stderr, "debugfs %s (%s)\n", E2FSPROGS_VERSION,
-		 E2FSPROGS_DATE);
+	fprintf (stderr, "%s %s (%s)\n", debug_prog_name, 
+		 E2FSPROGS_VERSION, E2FSPROGS_DATE);
 
 	while ((c = getopt (argc, argv, "iwcR:f:b:s:Vd:")) != EOF) {
 		switch (c) {
@@ -1851,7 +1856,7 @@ int main(int argc, char **argv)
 				error_message(EXT2_ET_BASE));
 			exit(0);
 		default:
-			com_err(argv[0], 0, usage);
+			com_err(argv[0], 0, usage, debug_prog_name);
 			return 1;
 		}
 	}
@@ -1860,7 +1865,7 @@ int main(int argc, char **argv)
 				superblock, blocksize, catastrophic,
 				data_filename);
 	
-	sci_idx = ss_create_invocation("debugfs", "0.0", (char *) NULL,
+	sci_idx = ss_create_invocation(debug_prog_name, "0.0", (char *) NULL,
 				       &debug_cmds, &retval);
 	if (retval) {
 		ss_perror(sci_idx, retval, "creating invocation");
@@ -1873,6 +1878,12 @@ int main(int argc, char **argv)
 		ss_perror(sci_idx, retval, "adding standard requests");
 		exit (1);
 	}
+	if (extra_cmds)
+		ss_add_request_table (sci_idx, extra_cmds, 1, &retval);
+	if (retval) {
+		ss_perror(sci_idx, retval, "adding extra requests");
+		exit (1);
+	}
 	if (request) {
 		retval = 0;
 		retval = ss_execute_line(sci_idx, request);
Signature · 1 line
1.5.3.1.19.gb5ef6
#2 lore raw

RFC PATCH 2/4 Don't byte swap extents information in the inode

Responsibility for byte swapping the extents information rests with the low-level extent code, which translates the on-disk extents information to the abstract extent format. The on-disk format will eventually get more complicated, in order to add support for 64-bit block numbers, bit-compressed extents, etc. So to avoid needing to expose all of that complexity in swapfs.c, the in-memory contents of i_blocks will not be byte-swapped and will be identical to the on-disk format.

---
 lib/ext2fs/swapfs.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)
lib/ext2fs/swapfs.c +4 -0
index bd0844d..52ca6a0 100644
@@ -158,7 +158,11 @@ void ext2fs_swap_inode_full(ext2_filsys fs, struct ext2_inode_large *t,
 	if (!hostorder)
 		has_data_blocks = ext2fs_inode_data_blocks(fs, 
 					   (struct ext2_inode *) t);
+	if (hostorder && (f->i_flags & EXT4_EXTENTS_FL))
+		has_data_blocks = 0;
 	t->i_flags = ext2fs_swab32(f->i_flags);
+	if (hostorder && (t->i_flags & EXT4_EXTENTS_FL))
+		has_data_blocks = 0;
 	t->i_dir_acl = ext2fs_swab32(f->i_dir_acl);
 	if (!islnk || has_data_blocks ) {
 		for (i = 0; i < EXT2_N_BLOCKS; i++)
Signature · 1 line
1.5.3.1.19.gb5ef6
#3 lore raw

RFC PATCH 3/4 e2fsck: factor out code to clear an inode into e2fsck_clear_inode()

Factor out code to clear a bogus inode and update e2fsck's internal data structures accordingly into a common routine, e2fsck_clear_inode(). This saves about 200 bytes in the compiled x86 e2fsck executable, and makes the code more maintainable in the long-term.

---
 e2fsck/e2fsck.h |    3 ++
 e2fsck/pass1.c  |   61 ++++++++++++++++++++++++++++--------------------------
 e2fsck/pass1b.c |    6 +----
 e2fsck/pass2.c  |    9 +-------
 e2fsck/pass4.c  |    7 +-----
 5 files changed, 38 insertions(+), 48 deletions(-)
e2fsck/e2fsck.h +3 -0
index 99c0ae5..d8e85ad 100644
@@ -430,6 +430,9 @@ extern int e2fsck_pass1_check_device_inode(ext2_filsys fs,
 					   struct ext2_inode *inode);
 extern int e2fsck_pass1_check_symlink(ext2_filsys fs,
 				      struct ext2_inode *inode, char *buf);
+extern void e2fsck_clear_inode(e2fsck_t ctx, ext2_ino_t ino, 
+			       struct ext2_inode *inode, int restart_flag,
+			       const char *source);
 
 /* pass2.c */
 extern int e2fsck_process_bad_inode(e2fsck_t ctx, ext2_ino_t dir,
e2fsck/pass1.c +32 -29

87 lines not on this page · raw message

e2fsck/pass1b.c +1 -5
index 118f956..5d062ca 100644
@@ -594,16 +594,13 @@ static void delete_file(e2fsck_t ctx, ext2_ino_t ino,
 						     delete_file_block, &pb);
 	if (pctx.errcode)
 		fix_problem(ctx, PR_1B_BLOCK_ITERATE, &pctx);
-	ext2fs_unmark_inode_bitmap(ctx->inode_used_map, ino);
-	ext2fs_unmark_inode_bitmap(ctx->inode_dir_map, ino);
 	if (ctx->inode_bad_map)
 		ext2fs_unmark_inode_bitmap(ctx->inode_bad_map, ino);
 	ext2fs_inode_alloc_stats2(fs, ino, -1, LINUX_S_ISDIR(inode.i_mode));
 
 	/* Inode may have changed by block_iterate, so reread it */
 	e2fsck_read_inode(ctx, ino, &inode, "delete_file");
-	inode.i_links_count = 0;
-	inode.i_dtime = ctx->now;
+	e2fsck_clear_inode(ctx, ino, &inode, 0, "delete_file");
 	if (inode.i_file_acl &&
 	    (fs->super->s_feature_compat & EXT2_FEATURE_COMPAT_EXT_ATTR)) {
 		count = 1;
@@ -629,7 +626,6 @@ static void delete_file(e2fsck_t ctx, ext2_ino_t ino,
 			delete_file_block(fs, &inode.i_file_acl,
 					  BLOCK_COUNT_EXTATTR, 0, 0, &pb);
 	}
-	e2fsck_write_inode(ctx, ino, &inode, "delete_file");
 }
 
 struct clone_struct {
e2fsck/pass2.c +1 -8
index 7f7635f..8e651c7 100644
@@ -1109,11 +1109,8 @@ static void deallocate_inode(e2fsck_t ctx, ext2_ino_t ino, char* block_buf)
 	struct problem_context	pctx;
 	__u32			count;
 	
-	ext2fs_icount_store(ctx->inode_link_info, ino, 0);
 	e2fsck_read_inode(ctx, ino, &inode, "deallocate_inode");
-	inode.i_links_count = 0;
-	inode.i_dtime = ctx->now;
-	e2fsck_write_inode(ctx, ino, &inode, "deallocate_inode");
+	e2fsck_clear_inode(ctx, ino, &inode, 0, "deallocate_inode");
 	clear_problem_context(&pctx);
 	pctx.ino = ino;
 
@@ -1121,10 +1118,6 @@ static void deallocate_inode(e2fsck_t ctx, ext2_ino_t ino, char* block_buf)
 	 * Fix up the bitmaps...
 	 */
 	e2fsck_read_bitmaps(ctx);
-	ext2fs_unmark_inode_bitmap(ctx->inode_used_map, ino);
-	ext2fs_unmark_inode_bitmap(ctx->inode_dir_map, ino);
-	if (ctx->inode_bad_map)
-		ext2fs_unmark_inode_bitmap(ctx->inode_bad_map, ino);
 	ext2fs_inode_alloc_stats2(fs, ino, -1, LINUX_S_ISDIR(inode.i_mode));
 
 	if (inode.i_file_acl &&
e2fsck/pass4.c +1 -6
index 0fb8ee7..3eca1c2 100644
@@ -43,17 +43,12 @@ static int disconnect_inode(e2fsck_t ctx, ext2_ino_t i)
 	if (!inode.i_blocks && (LINUX_S_ISREG(inode.i_mode) ||
 				LINUX_S_ISDIR(inode.i_mode))) {
 		if (fix_problem(ctx, PR_4_ZERO_LEN_INODE, &pctx)) {
-			ext2fs_icount_store(ctx->inode_link_info, i, 0);
-			inode.i_links_count = 0;
-			inode.i_dtime = ctx->now;
-			e2fsck_write_inode(ctx, i, &inode,
+			e2fsck_clear_inode(ctx, i, &inode, 0, 
 					   "disconnect_inode");
 			/*
 			 * Fix up the bitmaps...
 			 */
 			e2fsck_read_bitmaps(ctx);
-			ext2fs_unmark_inode_bitmap(ctx->inode_used_map, i);
-			ext2fs_unmark_inode_bitmap(ctx->inode_dir_map, i);
 			ext2fs_inode_alloc_stats2(fs, i, -1,
 						  LINUX_S_ISDIR(inode.i_mode));
 			return 0;
Signature · 1 line
1.5.3.1.19.gb5ef6
#4 lore raw

RFC PATCH 4/4 Initial checkin for new extents API

Initial implemenation of extents support for e2fsprogs. This is enough so that e2fsck will pass a valid filesystem containing extents.

Eventually this will be split into a patch for the library changes, and a separate patch for e2fsck, followed by test cases for e2fsprogs.

---
 e2fsck/message.c          |    2 +
 e2fsck/pass1.c            |   86 ++++++-
 e2fsck/problem.c          |    5 +
 e2fsck/problem.h          |    3 +
 lib/ext2fs/Makefile.in    |   68 +++++-
 lib/ext2fs/ext2_err.et.in |   40 +++-
 lib/ext2fs/ext2fs.h       |   62 +++++-
 lib/ext2fs/ext3_extents.h |   20 ++
 lib/ext2fs/extent.c       |  616 +++++++++++++++++++++++++++++++++++++++++++++
 lib/ext2fs/extent_dbg.ct  |   32 +++
 10 files changed, 926 insertions(+), 8 deletions(-)
 create mode 100644 lib/ext2fs/extent.c
 create mode 100644 lib/ext2fs/extent_dbg.ct
e2fsck/message.c +2 -0
index b2e3e0f..accf3a0 100644
@@ -80,6 +80,7 @@
  * 	@S	superblock
  * 	@u	unattached
  * 	@v	device
+ *	@x	extent
  * 	@z	zero-length
  */
 
@@ -134,6 +135,7 @@ static const char *abbrevs[] = {
 	N_("Ssuper@b"),
 	N_("uunattached"),
 	N_("vdevice"),
+	N_("xextent"),
 	N_("zzero-length"),
 	"@@",
 	0
e2fsck/pass1.c +82 -4

104 lines not on this page · raw message

e2fsck/problem.c +5 -0
index 0b6fd39..8c8be7e 100644
@@ -784,6 +784,11 @@ static struct e2fsck_problem problem_table[] = {
 	  N_("@i %i is a %It but it looks like it is really a directory.\n"),
 	  PROMPT_FIX, 0 },
 
+	/* Error while reading extent tree */
+	{ PR_1_READ_EXTENT,
+	  N_("Error while reading over @x tree in @i %i: %m\n"),
+	  PROMPT_CLEAR_INODE, 0 },
+
 	/* Pass 1b errors */
 
 	/* Pass 1B: Rescan for duplicate/bad blocks */
e2fsck/problem.h +3 -0
index f5f7212..538dee2 100644
@@ -455,6 +455,9 @@ struct problem_context {
 /* inode appears to be a directory */
 #define PR_1_TREAT_AS_DIRECTORY		0x010055
 
+/* Error while reading extent tree */
+#define PR_1_READ_EXTENT		0x010056
+
 /*
  * Pass 1b errors
  */
lib/ext2fs/Makefile.in +67 -1

98 lines not on this page · raw message

lib/ext2fs/ext2_err.et.in +38 -2
index eda4bb4..5d62548 100644
@@ -62,8 +62,8 @@ ec	EXT2_ET_MAGIC_E2IMAGE,
 ec	EXT2_ET_MAGIC_INODE_IO_CHANNEL,
 	"Wrong magic number for inode io_channel structure"
 
-ec	EXT2_ET_MAGIC_RESERVED_9,
-	"Wrong magic number --- RESERVED_9"
+ec	EXT2_ET_MAGIC_EXTENT_HANDLE,
+	"Wrong magic number for ext4 extent handle"
 
 ec	EXT2_ET_BAD_MAGIC,
 	"Bad magic number in super-block"
@@ -326,5 +326,41 @@ ec	EXT2_ET_TDB_ERR_NOEXIST,
 ec	EXT2_ET_TDB_ERR_RDONLY,
 	"TDB: Write not permitted"
 
+ec	EXT2_ET_EXTENT_HEADER_BAD,
+	"Corrupt extent header"
+
+ec	EXT2_ET_EXTENT_INDEX_BAD,
+	"Corrupt extent index"
+
+ec	EXT2_ET_EXTENT_LEAF_BAD,
+	"Corrupt extent"
+
+ec	EXT2_ET_EXTENT_NO_SPACE,
+	"No free space in extent map"
+
+ec	EXT2_ET_INODE_NOT_EXTENT,
+	"Inode does not use extents"
+
+ec	EXT2_ET_EXTENT_NO_NEXT,
+	"No 'next' extent"
+
+ec	EXT2_ET_EXTENT_NO_PREV,
+	"No 'previous' extent"
+
+ec	EXT2_ET_EXTENT_NO_UP,
+	"No 'up' extent"
+
+ec	EXT2_ET_EXTENT_NO_DOWN,
+	"No 'down' extent"
+
+ec	EXT2_ET_NO_CURRENT_NODE,
+	"No current node"
+
+ec	EXT2_ET_OP_NOT_SUPPORTED,
+	"Ext2fs operation not supported"
+
+ec	EXT2_ET_CANT_INSERT_EXTENT,
+	"No room to insert extent in node"
+
 	end
 
lib/ext2fs/ext2fs.h +61 -1

86 lines not on this page · raw message

lib/ext2fs/ext3_extents.h +20 -0
index c4178c9..ba980e4 100644
@@ -126,6 +126,26 @@ struct ext3_ext_path {
 #define EXT_MAX_BLOCK	0xffffffff
 #define EXT_CACHE_MARK	0xffff
 
+/*
+ * EXT_INIT_MAX_LEN is the maximum number of blocks we can have in an
+ * initialized extent. This is 2^15 and not (2^16 - 1), since we use the
+ * MSB of ee_len field in the extent datastructure to signify if this
+ * particular extent is an initialized extent or an uninitialized (i.e.
+ * preallocated).
+ * EXT_UNINIT_MAX_LEN is the maximum number of blocks we can have in an
+ * uninitialized extent.
+ * If ee_len is <= 0x8000, it is an initialized extent. Otherwise, it is an
+ * uninitialized one. In other words, if MSB of ee_len is set, it is an
+ * uninitialized extent with only one special scenario when ee_len = 0x8000.
+ * In this case we can not have an uninitialized extent of zero length and
+ * thus we make it as a special case of initialized extent with 0x8000 length.
+ * This way we get better extent-to-group alignment for initialized extents.
+ * Hence, the maximum number of blocks we can have in an *initialized*
+ * extent is 2^15 (32768) and in an *uninitialized* extent is 2^15-1 (32767).
+ */
+#define EXT_INIT_MAX_LEN	(1UL << 15)
+#define EXT_UNINIT_MAX_LEN	(EXT_INIT_MAX_LEN - 1)
+
 
 #define EXT_FIRST_EXTENT(__hdr__) \
 	((struct ext3_extent *) (((char *) (__hdr__)) +		\
lib/ext2fs/extent.c new file +616 -0

616 lines not on this page · raw message

lib/ext2fs/extent_dbg.ct new file +32 -0
new file mode 100644
index 0000000..21ece39
@@ -0,0 +1,32 @@
+#
+# Copyright (C) 1993 Theodore Ts'o.  This file may be redistributed
+# under the terms of the GNU Public License.
+#
+command_table extent_cmds;
+
+request do_inode, "Open an inode",
+	inode;
+
+request do_current_node, "Current extent node",
+	current_node, current;
+
+request do_root_node, "Goto root extent",
+	root_node, root;
+
+request do_first_sib, "First sibling",
+	first_sibling, first_sib;
+
+request do_next_sib, "Next sibling",
+	next_sibling, next_sib, next;
+
+request do_prev_sib, "Previous sibling",
+	prev_sibling, prev_sib, prev;
+
+request do_up, "Up node",
+	up_node, up;
+
+request do_down, "Down node",
+	down_node, down;
+
+end;
+
Signature · 1 line
1.5.3.1.19.gb5ef6