e2fsprogs: allocate inode table wholly within group

Building e2fsprogs 1.42.8 on ppc, I got this:

r_1024_small_bg: ext2 1024 blocksize with small block groups: failed

Because during the resize step it did this:

Itable move group 1 block 1030->1092 (diff 62)

but during e2fsck it found:

/tmp/e2fsprogs-tmp.uiFhgP: Inode table for group 1 is not in group. (block 1092)

i.e. from dumpe2fs we can see:

Group 1: (Blocks 1025-1110)
Backup superblock at 1025, Group descriptors at 1026-1026 Block bitmap at 1090 (+65), Inode bitmap at 1091 (+66) Inode table at 1092-1123 (+67)
^^^^ beyond end of block group

ext2fs_allocate_group_table() currently sends the last block of the group as an acceptable *starting* point for the inode table allocation.

Because the inode table may be several blocks, and must reside wholly within the group, move the last acceptable starting block back by this amount so that the allocated range cannot extend past the end of the group.

Signed-off-by Eric Sandeen <[email protected]>

---

p.s. I *think* this fixes it; I've not convinced myself that the failure is 100% deterministic, so extra eyeballs on the patch are welcome.

Thanks,
-Eric

Index: e2fsprogs-1.42.8/lib/ext2fs/alloc_tables.c
===================================================================

lib/ext2fs/alloc_tables.c +3 -1

@@ -200,6 +200,9 @@ errcode_t ext2fs_allocate_group_table(ex
 	}
 
 	if (!ext2fs_inode_table_loc(fs, group)) {
+		/* Inode table must fit within group, so adjust last blk */
+		last_blk -= (fs->inode_blocks_per_group - 1);
+
 		retval = ext2fs_get_free_blocks2(fs, group_blk, last_blk,
 						fs->inode_blocks_per_group,
 						bmap, &new_blk);
 
--

To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Building e2fsprogs 1.42.8 on ppc, I got this:

r_1024_small_bg: ext2 1024 blocksize with small block groups: failed

Because during the resize step it did this:

Itable move group 1 block 1030->1092 (diff 62)

but during e2fsck it found:

/tmp/e2fsprogs-tmp.uiFhgP: Inode table for group 1 is not in group. (block 1092)

i.e. from dumpe2fs we can see:

Group 1: (Blocks 1025-1110)
Backup superblock at 1025, Group descriptors at 1026-1026 Block bitmap at 1090 (+65), Inode bitmap at 1091 (+66) Inode table at 1092-1123 (+67)
^^^^ beyond end of block group

There seems to be something wrong here. The test file system was created like this:

mke2fs -t ext2 -O ^resize_inode -b 1024 -g 1024 -qF /tmp/foo.img 64M

The file system hence should have 64 block groups, and dumpe2fs before the resize looks like this on an x86 system:

Group 1: (Blocks 1025-2048)
Backup superblock at 1025, Group descriptors at 1026-1027 Block bitmap at 1028 (+3), Inode bitmap at 1029 (+4) Inode table at 1030-1061 (+5)

... and after:

Group 1: (Blocks 1025-2048)
Backup superblock at 1025, Group descriptors at 1026-1089 Block bitmap at 1090 (+65), Inode bitmap at 1091 (+66) Inode table at 1092-1123 (+67)

Note the range of block group #1: 1025-2048, whereas on the PPC, apparently the range is quite different: Group 1: (Blocks 1025-1110)

So there's something else going really wrong here....

Quoted 12 lines · Eric Sandeen

Building e2fsprogs 1.42.8 on ppc, I got this:

r_1024_small_bg: ext2 1024 blocksize with small block groups: failed

Because during the resize step it did this:

Itable move group 1 block 1030->1092 (diff 62)

but during e2fsck it found:

/tmp/e2fsprogs-tmp.uiFhgP: Inode table for group 1 is not in group. (block 1092)

Quoted 7 lines

i.e. from dumpe2fs we can see:

Group 1: (Blocks 1025-1110)
Backup superblock at 1025, Group descriptors at 1026-1026 Block bitmap at 1090 (+65), Inode bitmap at 1091 (+66) Inode table at 1092-1123 (+67)
^^^^ beyond end of block group

There seems to be something wrong here. The test file system was created like this:

mke2fs -t ext2 -O ^resize_inode -b 1024 -g 1024 -qF /tmp/foo.img 64M

The file system hence should have 64 block groups, and dumpe2fs before the resize looks like this on an x86 system:

Group 1: (Blocks 1025-2048)
Backup superblock at 1025, Group descriptors at 1026-1027 Block bitmap at 1028 (+3), Inode bitmap at 1029 (+4) Inode table at 1030-1061 (+5)

... and after:

Group 1: (Blocks 1025-2048)
Backup superblock at 1025, Group descriptors at 1026-1089 Block bitmap at 1090 (+65), Inode bitmap at 1091 (+66) Inode table at 1092-1123 (+67)

Note the range of block group #1: 1025-2048, whereas on the PPC, apparently the range is quite different: Group 1: (Blocks 1025-1110)

So there's something else going really wrong here....

- Ted

Quoted 12 lines · Eric Sandeen

Building e2fsprogs 1.42.8 on ppc, I got this:

r_1024_small_bg: ext2 1024 blocksize with small block groups: failed

Because during the resize step it did this:

Itable move group 1 block 1030->1092 (diff 62)

but during e2fsck it found:

/tmp/e2fsprogs-tmp.uiFhgP: Inode table for group 1 is not in group. (block 1092)

Quoted 7 lines

i.e. from dumpe2fs we can see:

Group 1: (Blocks 1025-1110)
Backup superblock at 1025, Group descriptors at 1026-1026 Block bitmap at 1090 (+65), Inode bitmap at 1091 (+66) Inode table at 1092-1123 (+67)
^^^^ beyond end of block group

There seems to be something wrong here. The test file system was created like this:

mke2fs -t ext2 -O ^resize_inode -b 1024 -g 1024 -qF /tmp/foo.img 64M

The file system hence should have 64 block groups, and dumpe2fs before the resize looks like this on an x86 system:

Group 1: (Blocks 1025-2048)
Backup superblock at 1025, Group descriptors at 1026-1027 Block bitmap at 1028 (+3), Inode bitmap at 1029 (+4) Inode table at 1030-1061 (+5)

... and after:

Group 1: (Blocks 1025-2048)
Backup superblock at 1025, Group descriptors at 1026-1089 Block bitmap at 1090 (+65), Inode bitmap at 1091 (+66) Inode table at 1092-1123 (+67)

Note the range of block group #1: 1025-2048, whereas on the PPC, apparently the range is quite different: Group 1: (Blocks 1025-1110)

So there's something else going really wrong here....

test_description="ext2 1024 blocksize with small block groups" FEATURES="-t ext2 -O ^resize_inode -b 1024 -g 1024" SIZE_1=64M
SIZE_2=2G
LOG=$test_name.log
E2FSCK=../e2fsck/e2fsck

. $cmd_dir/scripts/resize_test

../resize/resize2fs -d 31 -M /tmp/e2fsprogs-tmp.gB4G7E 2G resize2fs 1.42.8 (20-Jun-2013)
forcing to min size 1121
Resizing the filesystem on /tmp/e2fsprogs-tmp.gB4G7E to 1121 (1k) blocks.

The actual problem seems to be that the test does successive "-M" minimal resizes, and eventually we resize into the middle of an inode table, leaving the end of the table beyond the fs.

Point "resize2fs -M" at the attached image once or twice w/ fscks in between and you should see it.

It seems like the obvious fix would be to move the inode table if necessary, as with the following patch.

Quoted 4 lines · Eric Sandeen

The actual problem seems to be that the test does successive "-M" minimal resizes, and eventually we resize into the middle of an inode table, leaving the end of the table beyond the fs.

Point "resize2fs -M" at the attached image once or twice w/ fscks in between and you should see it.

I've been going through my patch backlog, so I finally had a chance to take a very close look at your test image. I now understand why things are failing.

1) The test image (which you said was generated on a ppc e2fsprogs?) was doing something very weird as far as the location of the allocation bitmaps and inode table:

Filesystem features:      ext_attr dir_index filetype sparse_super
Inode count:              512
Block count:              1247

...

Group 0: (Blocks 1-1024)
Primary superblock at 1, Group descriptors at 2-2 Block bitmap at 66 (+65), Inode bitmap at 67 (+66) Inode table at 68-99 (+67)

Group 1: (Blocks 1025-1246)
Backup superblock at 1025, Group descriptors at 1026-1026 Block bitmap at 1090 (+65), Inode bitmap at 1091 (+66) Inode table at 1092-1123 (+67)

Compare and contrast this with what x86 and Debian's ppc mke2fs creates:

Group 0: (Blocks 1-1024)
Primary superblock at 1, Group descriptors at 2-2 Block bitmap at 3 (+2), Inode bitmap at 4 (+3) Inode table at 5-14 (+4)

Group 1: (Blocks 1025-1246)
Backup superblock at 1025, Group descriptors at 1026-1026 Block bitmap at 1027 (+2), Inode bitmap at 1028 (+3) Inode table at 1029-1038 (+4)

So I'm not sure why Fedora's ppc mke2fs is creating file systems in this way, but that's one of the causes of the bug.

2) The second cause of the bug is that calculate_minimum_resize_size(), when we calculate the number of blocks for the last group, the code has an implicit assumption that the metadata blocks are located at the very beginning of the block
group. That's an easy fix.

Quoted 2 lines

It seems like the obvious fix would be to move the inode table if necessary, as with the following patch.

Your patch is a good one, but at least in the context of resize2fs -M, we should be fixing calculate_minimum_resize_size() so we can avoid needing to move the inode table (since even if it can succeed, it's not worth the danger).

I'll send out some patches to address this. Thanks for sending the test image; and my apologies for not having time to get back to this until now.

- Ted

Quoted 6 lines · Theodore Ts'o

Quoted 4 lines · Eric Sandeen

The actual problem seems to be that the test does successive "-M" minimal resizes, and eventually we resize into the middle of an inode table, leaving the end of the table beyond the fs.

Point "resize2fs -M" at the attached image once or twice w/ fscks in between and you should see it.

Quoted 7 lines

I've been going through my patch backlog, so I finally had a chance to take a very close look at your test image. I now understand why things are failing.

1) The test image (which you said was generated on a ppc e2fsprogs?) was doing something very weird as far as the location of the allocation bitmaps and inode table:

Yes, this was just during a fedora build, during the "make check" phase.

https://bugzilla.redhat.com/show_bug.cgi?id=980519

No idea why things should be coming out differently, that's a bit alarming in and of itself.

(Fedora isn't carrying any interesting patches to speak of).

Quoted 15 lines · Eric Sandeen

Quoted 6 lines · Theodore Ts'o

Quoted 4 lines · Eric Sandeen

The actual problem seems to be that the test does successive "-M" minimal resizes, and eventually we resize into the middle of an inode table, leaving the end of the table beyond the fs.

Point "resize2fs -M" at the attached image once or twice w/ fscks in between and you should see it.

Quoted 7 lines

I've been going through my patch backlog, so I finally had a chance to take a very close look at your test image. I now understand why things are failing.

1) The test image (which you said was generated on a ppc e2fsprogs?) was doing something very weird as far as the location of the allocation bitmaps and inode table:

Quoted 8 lines

Yes, this was just during a fedora build, during the "make check" phase.

https://bugzilla.redhat.com/show_bug.cgi?id=980519

No idea why things should be coming out differently, that's a bit alarming in and of itself.

(Fedora isn't carrying any interesting patches to speak of).

But I am doing this:

%check

(quoted diff) +8 -0

+# XXX ERS Hack for now; this bug has existed for a while,
+# i.e. it is not a regression in this release, but there
+# is no fix yet, and we need to get this package building.
+# See Bug 987133 - resize2fs tests failing on ppc, s390
+rm -rf tests/r_1024_small_bg*
+rm -rf tests/r_64bit_big_expand*
+rm -rf tests/r_bigalloc_big_expand*
+rm -rf tests/r_ext4_big_expand*
 make check

I'll retest w/ your patches, thanks.