more resize breakage

After considering how many resize2fs corruptions we've had, I decided to try to write a resize fuzzer which picks random parameters and sizes, and sees what happens with online & offline grow & offline shrink. When I get it cleaner, I'll send it out to play with.

But it is indeed finding resize issues; for example, with e2fsprogs git master & v3.15-rc3,

# truncate --size=11g fsfile
# mke2fs -t ext4 -O 64bit,^bigalloc,extent,^flex_bg,^meta_bg,resize_inode,sparse_super,^uninit_bg, -E packed_meta_blocks=0 -b 1024 -I 512 -g 3648 fsfile
mke2fs 1.43-WIP (18-May-2014)
Discarding device blocks: done                            
Creating filesystem with 11534336 1k blocks and 708288 inodes
Filesystem UUID: 25a71d26-0a54-4732-bcb5-d08bdb0878ab
Superblock backups stored on blocks: 
	3649, 10945, 18241, 25537, 32833, 91201, 98497, 178753, 295489, 456001, 
	886465, 1251265, 2280001, 2659393, 7978177, 8758849, 11400001

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

# truncate --size=35g fsfile
# mount -o loop fsfile mnt/
# resize2fs /dev/loop0
resize2fs 1.43-WIP (18-May-2014)
Filesystem at /dev/loop0 is mounted on /mnt/test2/resizefuzzer/mnt; on-line resizing required
old_desc_blocks = 198, new_desc_blocks = 629
The filesystem on /dev/loop0 is now 36700160 blocks long.

# umount mnt
# e2fsck -fn fsfile 
e2fsck 1.43-WIP (18-May-2014)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Block bitmap differences:  +54721 +(54723--54732) +54734 +54736 +54741 +(54743--54745) +(54747--54752) +(54755--54764) +54766 +54768 +54773 +(54775--54777) +(54779--54786) +(54788--54796) +54798 +54800 +54805 +(54807--54809) +(54811--54816) +54818 +(54821--54822) +54826 +(54829--54834) -(54838--54840) -54867 -55234 -55239 -(55242--55245) -55247 -(55249--55252) -55254 -55258 -(55265--55266) -55271 -(55274--55277) -55279 -(55281--55284) -55286 -55290 -55299 -55303 -(55306--55309) -55311 -(55313--55316) -55318 -55322 -55329 -(55331--55332) -(55335--55337) -(55339--55340) -(55350--55352) -55379 -55745 -55752 -(55755--55756) -(55758--55759) -(55761--55764) -55766 -55770 -55778 -55784 -(55787--55788) -(55790--55791) -(55793--55796) -55798 -55802 -(55809--55810) -55816 -(55819--55820) -(55822--55
 823) -(55825--55828) -55830 -55834 -(55842--55844) -(55847--55849) -(55851--55852) -(55862--55864) -55891 -56257 -(56263--56264) -56266 -(56268--56271) -(56273--56276) -56278 -56282 -56290 !
 -(56295--5
6296) -56298 -(56300--56303) -(56305--56308) -56310 -56314 -(56321--56322) -(56327--56328) -56330 -(56332--56335) -(56337--56340) -56342 -56346 -(56354--56356) -(56359--56361) -(56363--56364) -(56374--56376) -56403 -56769 -56777 -56780 -(56784--56788) -56790 -56794 -56802 -56809 -56812 -(56816--56820) -56822 -56826 -(56833--56834) -56841 -56844 -(56848--56852) -56854 -56858 -(56866--56868) -(56871--56873) -(56875--56876) -(56886--56888) -56915 -57281 -57287 -(57289--57291) -57293 -(57296--57300) -57302 -57306 -57314 -57319 -(57321--57323) -57325 -(57328--57332) -57334 -57338 -(57345--57346) -57351 -(57353--57355) -57357 -(57360--57364) -57366 -57370 -(57378--57380) -(57383--57385) -(57387--57388) -(57398--57400) -57427 -57793 -(57800--57801) -57803 -57806 -(57808--57812) -57814 -57818 -578
 26 -(57832--57833) -57835 -57838 -(57840--57844) -57846 -57850 -(57857--57858) -(57864--57865) -57867 -57870 -(57872--57876) -57878 -57882 -(57890--57892) -(57895--57897) -(57899--57900) -(!
 57910--579
12) -57939 -58305 -(58311--58314) -(58317--58318) -(58320--58324) -58326 -58330 -58338 -(58343--58346) -(58349--58350) -(58352--58356) -58358 -58362
Fix? no

Free blocks count wrong for group #15 (3534, counted=3280). Fix? no

Free blocks count wrong (35511254, counted=35511000). Fix? no

fsfile: ********** WARNING: Filesystem still has errors **********

fsfile: 11/2253664 files (0.0% non-contiguous), 1188906/36700160 blocks

Sad face. :(

I haven't looked into these failures, but it seems clear that "we're not there, yet."

Resize seems to be getting so fiendishly complex with all the format options it must cope with.

-Eric

55823) -(55825--55828) -55830 -55834 -(55842--55844) -(55847--55849) -(55851--55852) -(55862--55864) -55891 -56257 -(56263--56264) -56266 -(56268--56271) -(56273--56276) -56278 -56282 -56290 !

7826 -(57832--57833) -57835 -57838 -(57840--57844) -57846 -57850 -(57857--57858) -(57864--57865) -57867 -57870 -(57872--57876) -57878 -57882 -(57890--57892) -(57895--57897) -(57899--57900) -(!

D'oh!

/me wonders, is offline grow any better?

Also I "extended" fsfuzz to corrupt only metadata blocks and made the kernel+e2fsck chew through all that crap. The kernel survived, but e2fsck seemed to die either failing to allocate blocks to resurrect the journal (bad bbitmap) or because of that thing where calling block_iterate on an inline data file makes e2fsck abort.

So, uh, ... long live the patchbomb? :(

--D

<snip>

Yes, offline passed.

yeah. Maybe I (you?) should try my testcase w/ your latest patchbomb. ;)

-Eric

I think I only have patches out for review for e2fsprogs at the moment. I've not put my grubby hands on kernel code in a while.

--D

Darrick,

I don't think the problem which Eric reported in the head of this thread is fixed by one of your patches, but you did send some patches which I think would fix some resize bugs, which are setting on the ext4 "dev" branch for the next merge window.

It's too late for me to deal with this tonight, but I'll run a quick check and see if Eric's repro fails with the tip of the ext4 patch queue.

Eric, thanks for creating this tool; I think it will incredibly helpful in terms of finding and fixing some of these edge cases.

Cheers,

- Ted