Skip to contentSkip to conversation

Fix resize2fs data/filesystem corruption bugs

PATCH In review Theodore Ts'o 2 participants 5 messages Apr 19, 2009 – Apr 21, 2009

lore
Share & tools
lore
#0 lore raw

cover Fix resize2fs data/filesystem corruption bugs

These patches fix some nasty resize2fs bugs when growing or shrinking ext4 filesystems off-line. Eric, you'll probably want get these patches into Fedora's e2fsprogs before F11 goes live.

The test case I've been using for these patches are as follows.

#!/bin/sh
mke2fs -i 8192 -b 1024 -t ext4 /dev/closure/testext4 1048576 mount /dev/closure/testext4 /mnt
cd /mnt
mkdir 1 2 3 4
cd 1; tar xjf /usr/projects/uml/shars/linux-2.6.11.tar.bz2; cd .. cd 2; tar xjf /usr/projects/uml/shars/linux-2.6.11.tar.bz2; cd .. cd 3; tar xjf /usr/projects/uml/shars/linux-2.6.11.tar.bz2; cd .. cd 4; tar xjf /usr/projects/uml/shars/linux-2.6.11.tar.bz2; cd .. cd /
umount /mnt
e2fsck -f /dev/closure/testext4

Then try resizing the filesystem smaller or larger. With e2fsprogs 1.41.4, "resize2fs -M -p /dev/closure/testext4" will just plain fail in the middle with the resize operation:

# /sbin/resize2fs  -M -p /dev/closure/testext4
resize2fs 1.41.4 (27-Jan-2009)
Resizing the filesystem on /dev/closure/textext4 to 863610 (1k) blocks.
Begin pass 2 (max = 116149)
Relocating blocks             XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Begin pass 3 (max = 128)
Scanning inode table          XXXXXXXXXXXXXXXXXXXX/sbin/resize2fs: No space left on device while trying to resize /dev/closure/textext4

The bugs fixed include potentially overwriting part of the existing inode table when shrinking the filesystem, and potentially overwriting in-use data blocks when growing the filesystem. So these are pretty serious errors, and means an e2fsprogs 1.41.5 will be coming out real soon now.

With these patches, I've resized the filesystem and then run a CRC checksum test across the unpacked 2.6.11 source tree. (Why 2.6.11? Because it was what I happened to have handy on my laptop at the time. :-)

- Ted

#1 lore raw

PATCH 1/3 resize2fs: Fix data corruption bug when growing an ext4 filesystem off-line

When allocating a new set of block group metadata as part of growing the filesystem, the resize2fs code assumes that the bitmap and inode table blocks are in their own block group; an assumption which is changed by the flex_bg feature. This commit works around the problem by temporarily turning off flex_bg while allocating the new block group metadata, to avoid potentially overwriting previously allocated data blocks.

---
 resize/resize2fs.c |   21 +++++++++++++++++++++
 1 files changed, 21 insertions(+), 0 deletions(-)
resize/resize2fs.c +21 -0
index 1173da1..0c1549b 100644
@@ -746,6 +746,7 @@ static errcode_t blocks_to_move(ext2_resize_t rfs)
 	errcode_t	retval;
 	ext2_filsys 	fs, old_fs;
 	ext2fs_block_bitmap	meta_bmap;
+	__u32		save_incompat_flag;
 
 	fs = rfs->new_fs;
 	old_fs = rfs->old_fs;
@@ -890,9 +891,29 @@ static errcode_t blocks_to_move(ext2_resize_t rfs)
 
 		/*
 		 * Allocate the missing data structures
+		 *
+		 * XXX We have a problem with FLEX_BG and off-line
+		 * resizing where we are growing the size of the
+		 * filesystem.  ext2fs_allocate_group_table() will try
+		 * to reserve the inode table in the desired flex_bg
+		 * location.  However, passing rfs->reserve_blocks
+		 * doesn't work since it only has reserved the blocks
+		 * that will be used in the new block group -- and
+		 * with flex_bg, we can and will allocate the tables
+		 * outside of the block group.  And we can't pass in
+		 * the fs->block_map because it doesn't handle
+		 * overlapping inode table movements right.  So for
+		 * now, we temporarily disable flex_bg to force
+		 * ext2fs_allocate_group_tables() to allocate the bg
+		 * metadata in side the block group, and the restore
+		 * it afterwards.  Ugly, until we can fix this up
+		 * right later.
 		 */
+		save_incompat_flag = fs->super->s_feature_incompat;
+		fs->super->s_feature_incompat &= ~EXT4_FEATURE_INCOMPAT_FLEX_BG;
 		retval = ext2fs_allocate_group_table(fs, i,
 						     rfs->reserve_blocks);
+		fs->super->s_feature_incompat = save_incompat_flag;
 		if (retval)
 			goto errout;
 
Signature · 1 line
1.5.6.3
#2 lore raw

PATCH 2/3 resize2fs: Fix data corruption bug when shrinking the inode table for ext4

If we need to shrink the inode table, we need to make sure the inodes contained in the part of the inode table we are vacating don't get reused as part of the filesystem shrink operation. This wasn't a problem with ext3 filesystems, since the inode table was located in the block group that was going away, so that location was not eligible for reallocation.

However with ext4 filesystems with flex_bg enabled, it's possible for a portion of the inode table in the last flex_bg group to be deallocated, but in a part of the filesystem which could be used as data blocks. So we must mark those blocks as reserved to prevent their reuse, and adjust the minimum filesystem size calculation to assure that we don't shrink a filesystem too small for the resize operation to succeed.

---
 resize/online.c    |    2 +-
 resize/resize2fs.c |   56 +++++++++++++++++++++++++++++++++++++++-------------
 resize/resize2fs.h |    1 +
 3 files changed, 44 insertions(+), 15 deletions(-)
resize/online.c +1 -1
index d581553..4bc5451 100644
@@ -104,7 +104,7 @@ errcode_t online_resize_fs(ext2_filsys fs, const char *mtpt,
 	 * but at least it allows on-line resizing to function.
 	 */
 	new_fs->super->s_feature_incompat &= ~EXT4_FEATURE_INCOMPAT_FLEX_BG;
-	retval = adjust_fs_info(new_fs, fs, *new_size);
+	retval = adjust_fs_info(new_fs, fs, 0, *new_size);
 	if (retval)
 		return retval;
 
resize/resize2fs.c +42 -14

115 lines not on this page · raw message

resize/resize2fs.h +1 -0
index ed25b06..fab7290 100644
@@ -128,6 +128,7 @@ extern errcode_t resize_fs(ext2_filsys fs, blk_t *new_size, int flags,
 					    unsigned long max));
 
 extern errcode_t adjust_fs_info(ext2_filsys fs, ext2_filsys old_fs,
+				ext2fs_block_bitmap reserve_blocks,
 				blk_t new_size);
 extern blk_t calculate_minimum_resize_size(ext2_filsys fs);
 
Signature · 1 line
1.5.6.3
#3 lore raw

PATCH 3/3 resize2fs: Fix corruption bug impacting ext4 filesystems with uninit_bg

Due to a fencepost bug, when skipping a block group whose block bitmap was uninitialized (and hence could not contain any blocks eligible for relaocation), the block immediately following the block group wasn't checked as well. If it was in use and required relocation, it wouldn't get properly relocated, with the result that an inode using such a block would end up, post resize, with a pointer to a block now outside the bounds of the filesystem.

This commit fixes this fencepost error.

---
 resize/resize2fs.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
resize/resize2fs.c +1 -1
index ac926ce..f3e7fd0 100644
@@ -804,7 +804,7 @@ static errcode_t blocks_to_move(ext2_resize_t rfs)
 			 * to the next block group.
 			 */
 			blk = ((g+1) * fs->super->s_blocks_per_group) +
-				fs->super->s_first_data_block;
+				fs->super->s_first_data_block - 1;
 			continue;
 		}
 		if (ext2fs_test_block_bitmap(old_fs->block_map, blk) &&
Signature · 1 line
1.5.6.3
#4 lore raw
In reply to Theodore Ts'o
Quoted 3 lines · Theodore Ts'o

These patches fix some nasty resize2fs bugs when growing or shrinking ext4 filesystems off-line. Eric, you'll probably want get these patches into Fedora's e2fsprogs before F11 goes live.

Thanks for the heads up Ted -

FWIW, the livecd creation process, which does a fair bit of offline resizing (shrinking), was indeed seeing corruption, and these patches do seem to resolve it. (I will soon be recommending that the final livecd image be fsck'd, as it gets subsequently dd'd to thousands of hard drives during the livecd install process!)

-Eric