kernel BUG at fs/ext4/extents.c:2738!

I have no idea what information you guys will need. Let me know and I'll to gather it.

Feb 20 19:26:11 storage kernel: ------------[ cut here ]------------
Feb 20 19:26:11 storage kernel: kernel BUG at fs/ext4/extents.c:2738!
Feb 20 19:26:11 storage kernel: invalid opcode: 0000 [#1] SMP 
Feb 20 19:26:11 storage kernel: last sysfs
file: /sys/devices/pci0000:00/0000:00:04.0/host0/target0:0:0/0:0:0:0/block/sda/sda2/dev
Feb 20 19:26:11 storage kernel: Dumping ftrace buffer:
Feb 20 19:26:11 storage kernel:   (ftrace buffer empty)
Feb 20 19:26:11 storage kernel: CPU 0 
Feb 20 19:26:11 storage kernel: Modules linked in: vboxnetflt vboxdrv

ext2 ext4 jbd2 crc16 fuse ipt_MASQUERADE iptable_nat nf_nat bridge stp nfsd lockd nfs_acl auth_rpcgss exportfs sunrpc bonding ipv6 dm_multipath raid456 async_xor async_memcpy async_tx xor kvm_amd kvm uinput usblp snd_hda_intel snd_seq_dummy cfi_cmdset_0002 jedec_probe snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss forcedeth snd_mixer_oss snd_pcm snd_timer snd_page_alloc snd_hwdep joydev sata_nv usb_storage cfi_probe gen_probe cfi_util serio_raw nvidia(P) snd firewire_ohci pcspkr k8temp hwmon ck804xrom soundcore firewire_core crc_itu_t mtd chipreg map_funcs i2c_nforce2 i2c_core ata_generic pata_acpi pata_amd [last unloaded: vboxdrv]

Feb 20 19:26:11 storage kernel: Pid: 24313, comm: mv Tainted: P        W
2.6.28.5.afb.fc10.3.x86_amd64 #1
Feb 20 19:26:11 storage kernel: RIP: 0010:[<ffffffffa0d41d30>]
[<ffffffffa0d41d30>] ext4_ext_get_blocks+0x185/0xe15 [ext4]
Feb 20 19:26:11 storage kernel: RSP: 0018:ffff880186f57818  EFLAGS:
00010202
Feb 20 19:26:11 storage kernel: RAX: 0000000000000060 RBX:
ffff88014d18c9c0 RCX: 0000000000000000
Feb 20 19:26:11 storage kernel: RDX: ffff880093880002 RSI:
0000000000000002 RDI: 000000000000000c
Feb 20 19:26:11 storage kernel: RBP: ffff880186f57958 R08:
ffffffffa0d52475 R09: 000000000000f30a
Feb 20 19:26:11 storage kernel: R10: ffff880093886000 R11:
0000000000000000 R12: ffff8801002151d0
Feb 20 19:26:11 storage kernel: R13: ffff880186f57b38 R14:
ffff880100215290 R15: 000000000000000c
Feb 20 19:26:11 storage kernel: FS:  00007f9ced51f780(0000)
GS:ffffffff8177b000(0000) knlGS:00000000ef88bb90
Feb 20 19:26:11 storage kernel: CS:  0010 DS: 0000 ES: 0000 CR0:
000000008005003b
Feb 20 19:26:11 storage kernel: CR2: 00007f12856bb000 CR3:
000000016ec63000 CR4: 00000000000006e0
Feb 20 19:26:11 storage kernel: DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
Feb 20 19:26:11 storage kernel: DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
Feb 20 19:26:11 storage kernel: Process mv (pid: 24313, threadinfo
ffff880186f56000, task ffff880100474500)
Feb 20 19:26:11 storage kernel: Stack:
Feb 20 19:26:11 storage kernel: 0000000086f57838 ffff880186f57b38
000000000000000c 0001500c00000000
Feb 20 19:26:11 storage kernel: 0000000000000000 ffff88014d18c9c0
ffffffff81094763 00011280ffffffff
Feb 20 19:26:11 storage kernel: ffff8801002151d0 ffff88023ca5c6c0
0000000000001000 0000000000000000
Feb 20 19:26:11 storage kernel: Call Trace:
Feb 20 19:26:11 storage kernel: [<ffffffff81094763>] ?
mempool_alloc_slab+0x11/0x13
Feb 20 19:26:11 storage kernel: [<ffffffff81094763>] ?
mempool_alloc_slab+0x11/0x13
Feb 20 19:26:11 storage kernel: [<ffffffff81349d1a>] ? _spin_lock_irq
+0x21/0x26
Feb 20 19:26:11 storage kernel: [<ffffffff813497f8>] ? __down_read
+0x3d/0xbd
Feb 20 19:26:11 storage kernel: [<ffffffff81094877>] ? mempool_alloc
+0x5b/0x113
Feb 20 19:26:11 storage kernel: [<ffffffffa0d32a0b>]
ext4_get_blocks_wrap+0x6f/0x239 [ext4]
Feb 20 19:26:11 storage kernel: [<ffffffffa0d331b1>] ext4_get_block
+0xb4/0xf2 [ext4]
Feb 20 19:26:11 storage kernel: [<ffffffff810eadc7>] do_mpage_readpage
+0x1a8/0x4cb
Feb 20 19:26:11 storage kernel: [<ffffffff8117737a>] ? radix_tree_insert
+0x186/0x1ca
Feb 20 19:26:11 storage kernel: [<ffffffff8109fe45>] ?
__inc_zone_page_state+0x25/0x27
Feb 20 19:26:11 storage kernel: [<ffffffff8109b6f1>] ? __lru_cache_add
+0x34/0x7c
Feb 20 19:26:11 storage kernel: [<ffffffff810eb1ed>] mpage_readpages
+0xb1/0xf4
Feb 20 19:26:11 storage kernel: [<ffffffffa0d330fd>] ? ext4_get_block
+0x0/0xf2 [ext4]
Feb 20 19:26:11 storage kernel: [<ffffffff81098a19>] ?
__alloc_pages_internal+0xfe/0x45d
Feb 20 19:26:11 storage kernel: [<ffffffffa0d330fd>] ? ext4_get_block
+0x0/0xf2 [ext4]
Feb 20 19:26:11 storage kernel: [<ffffffff8111acb2>] ?
ext3_ordered_write_end+0x110/0x127
Feb 20 19:26:11 storage kernel: [<ffffffffa0d31216>] ext4_readpages
+0x1a/0x1c [ext4]
Feb 20 19:26:11 storage kernel: [<ffffffff8109a888>]
__do_page_cache_readahead+0xf0/0x163
Feb 20 19:26:11 storage kernel: [<ffffffff8109abf9>] ondemand_readahead
+0x178/0x18a
Feb 20 19:26:11 storage kernel: [<ffffffff8109ac80>]
page_cache_async_readahead+0x75/0x82
Feb 20 19:26:11 storage kernel: [<ffffffff81093fb6>]
generic_file_aio_read+0x27e/0x58f
Feb 20 19:26:11 storage kernel: [<ffffffff810c60dc>] do_sync_read
+0xe7/0x12d
Feb 20 19:26:11 storage kernel: [<ffffffff81058821>] ?
autoremove_wake_function+0x0/0x38
Feb 20 19:26:11 storage kernel: [<ffffffff81349a1a>] ? _spin_lock
+0x9/0xc
Feb 20 19:26:11 storage kernel: [<ffffffff81142d30>] ?
security_file_permission+0x11/0x13
Feb 20 19:26:11 storage kernel: [<ffffffff810c6a59>] vfs_read+0xa8/0x102
Feb 20 19:26:11 storage kernel: [<ffffffff810c6b77>] sys_read+0x47/0x6e
Feb 20 19:26:11 storage kernel: [<ffffffff8101104a>]
system_call_fastpath+0x16/0x1b
Feb 20 19:26:11 storage kernel: Code: 66 41 8b 96 46 ff ff ff 48 8b 9d
e8 fe ff ff 0f b7 f2 48 63 c6 48 6b c0 30 48 8b 4c 18 10 48 85 c9 75 0c
85 f6 0f 84 58 08 00 00 <0f> 0b eb fe 44 8b 41 04 8b 39 44 8b 51 08 66
44 8b 49 06 66 41 
Feb 20 19:26:11 storage kernel: RIP  [<ffffffffa0d41d30>]
ext4_ext_get_blocks+0x185/0xe15 [ext4]
Feb 20 19:26:11 storage kernel: RSP <ffff880186f57818>
Feb 20 19:26:11 storage kernel: ---[ end trace 455a038508c771a2 ]---

since this isn't a stock fc10 kernel (?) i'm nto exactly sure what the BUG was, I assume it was:

        BUG_ON(path[depth].p_ext == NULL && depth != 0);

?

Can you reproduce this? Perhaps we can reproduce it from an e2image as well if so, then it could be investigated ....

(Or, if you can reproduce it, perhaps try a .29 kernel first)

- Eric

...I haven't noticed before, but before that message few more lines where recorded, which makes sense with the BUG line. Also, running fsck.ext4 on that filesystem I always got a Segment fault at an specific point, which it is impossible to fix. I'm in a process to remove files from that filesystem, as soon it finish I can run fsck again and send the result. Meanwhile, when removing files, rm stuck at a certain inode, which caused the first BUG message.

-- Andreas

/* output from rm */

removed #208971861/HardDisks/fedora_9_i386_03.vdi' removed #208971861/HardDisks/freebsd_i386_7.1-rc1.vdi' removed #208971861/HardDisks/fedora_10_i386_02.vdi' removed directory: #208971861/HardDisks' removed `#208971861/VirtualBox.xml'
removed directory: `#208971861' ^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C^C

/* from /var/log/messages just before the BUG message */

Feb 20 18:27:13 storage kernel: EXT4-fs error (device dm-4):
ext4_ext_find_extent: bad header in inode #208971873: unexpected
eh_depth - magic f30a, entries 1, max 340(0), depth 0(1)
Feb 20 18:27:13 storage kernel: EXT4-fs error (device dm-4):
ext4_ext_find_extent: bad header in inode #208971873: unexpected
eh_depth - magic f30a, entries 1, max 340(0), depth 0(1)
Feb 20 18:27:13 storage kernel: EXT4-fs error (device dm-4):
ext4_ext_find_extent: bad header in inode #208971873: unexpected
eh_depth - magic f30a, entries 1, max 340(0), depth 0(1)
Feb 20 18:27:13 storage kernel: EXT4-fs error (device dm-4):
ext4_ext_find_extent: bad header in inode #208971873: unexpected
eh_depth - magic f30a, entries 1, max 340(0), depth 0(1)
Feb 20 18:27:13 storage kernel: EXT4-fs error (device dm-4):
ext4_ext_find_extent: bad header in inode #208971873: unexpected
eh_depth - magic f30a, entries 1, max 340(0), depth 0(1)
Feb 20 18:27:13 storage kernel: EXT4-fs error (device dm-4):
ext4_ext_find_extent: bad header in inode #208971873: unexpected
eh_depth - magic f30a, entries 1, max 340(0), depth 0(1)
Feb 20 18:27:13 storage kernel: EXT4-fs error (device dm-4):
ext4_ext_find_extent: bad header in inode #208971873: unexpected
eh_depth - magic f30a, entries 1, max 340(0), depth 0(1)
Feb 20 18:27:13 storage kernel: EXT4-fs error (device dm-4):
ext4_ext_find_extent: bad header in inode #208971873: unexpected
eh_depth - magic f30a, entries 1, max 340(0), depth 0(1)
Feb 20 18:27:13 storage kernel: EXT4-fs error (device dm-4):

ext4_ext_find_extent: bad header in inode #208971873: unexpected eh_depth - magic f30a, entries 1, max 340(0), depth 0(1)

-----Original Message-----
From: Eric Sandeen <[email protected]>
To: Andreas Friedrich Berendsen <[email protected]>
Cc: [email protected]
Subject: Re: kernel BUG at fs/ext4/extents.c:2738!
Date: Fri, 20 Feb 2009 09:42:20 -0600

since this isn't a stock fc10 kernel (?) i'm nto exactly sure what the BUG was, I assume it was:

        BUG_ON(path[depth].p_ext == NULL && depth != 0);

?

Can you reproduce this? Perhaps we can reproduce it from an e2image as well if so, then it could be investigated ....

(Or, if you can reproduce it, perhaps try a .29 kernel first)

- Eric

please stop w/ the file removal for a moment, and see if you can make an e2image of the filesystem which reproduces the e2fsck problem ....

-Eric

rm was in a state which was impossible to stop (^C) or kill. I had to restart (init 6) my box.

Output of fsck is below. Now is much better than previously. At least fsck is finishing the process instead of generating a segment fault.

I have the original e2image before the rm but is a 87GB file.

# ll *store*
-rw-r--r-- 1 root root    41272136 2009-02-13 00:04 dumpe2fs_store_fs.txt
-rw------- 1 root root 93106847744 2009-02-13 04:39 e2image_store_fs.image
-rw-r--r-- 1 root root        1798 2009-02-13 00:05 tune2fs_store_fs.txt

/* fsck */

# fsck.ext4 -C 0 -D -F -y -v /dev/vgstore/lvol0
e2fsck 1.41.4 (27-Jan-2009)
/dev/vgstore/lvol0 contains a file system with errors, check forced. Pass 1: Checking inodes, blocks, and sizes
Deleted inode 32641213 has zero dtime. Fix? yes

Pass 2: Checking directory structure
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Block bitmap differences: -(379913--397499) -(397504--397507) -(397509--397612) -(397625--397639) -(398032--398037) -(398058--398061) -(398319--398320) -(398336--398351) -(398384--398387) -(398396--398401) -(398406--398413) -398783 -399143 -(399496--399498) -(399565--399570) -(399651--399662) -(399786--399801) -(399834--399848) -(399866--399873) -(399922--399929) -(399946--399951) -(399976--399983) -(399988--399989) -(399994--399999) -(400032--400046) -(400271--400293) -(400384--401644) -(401665--401668) -(402157--402178) -(402183--402220) -(402411--402412) -(402565--402586) -(402591--402623) -(402688--402703) -(403779--403785) -(404480--436981) -(926656--926719) -130556146 -203358338 -(353961600--353961727) -(353976896--353977023) -(354060288--354060543) -(354060608--354060671) -(3622191
24--362219187) -(412878208--412878335) -(424068646--424068709) -(430539904--430539967) -(430540020--430540095) -(430540160--430540223) -(430540288--430540543) -(727193772--727194042) -(727194112--727194155) -(727215616--727215743) -(727215808--727215935) -(727216000--727216127) -(727216192--727216255) -727222434 -(727243584--727243711) -(727243776--727243903) -(727244032--727244095) -(727244416--727244479) -(727256410--727256601) -(727268928--727269183) -(727269376--727269439) -(727269504--727269567) -(727287104--727287167) -(727292608--727292671) -(727293504--727293631) -(727293696--727293759) -(727314944--727315071) -(727315136--727315263) -(727315328--727315455) -(727316480--727316607) -(727316672--727316863) -(727317248--727317439) -(727317504--727317695) -(727400384--727400447) -(72 7414720--727414975) -(727415168--727415359) -(727415488--727415551) -(727427328--727427583) -(727427692--727427819) -(727427884--727427947) -(727428096--727428159) -(727428224--727428351) -(727428480--727428671) -(727688488--727688679) -(727778112--727778175) -(729037504--729037631) -(729037696--729037759) -(729037776--729037887) -(729038084--729038143) -(729038892--729038975) -729088537 -(729117400--729117407) -(729118208--729118271) -(729171000--729171007) -(729175232--729175339) -(729188616--729188647) -(729188696--729188767) -(729205964--729206015) -(729206192--729206207) -(729206644--729206655) -(729213696--729213823) -(729213946--729213951) -(729216668--729216695) -(729217280--729217343) -(729217536--729217599) -(729247992--729247999) -(729275809--729275835) -(729286208--729286335)
-(729286400--729286463) -(729308608--729308735) -(729315136--729315199) -(729315216--729315279) -(729315328--729315391) -(729315776--729315967) -(729316096--729316223) -(729316288--729316415) -(729489728--729489791) -(729490760--729490887) -(729587264--729587327) -(729587968--729588095) -(729716864--729717055) -(729744704--729744767) -(729746501--729746536) -(729778098--729778289) -(729780004--729780131) -(729845394--729845521) -(729846016--729846143) -(729849574--729849637) -(729849894--729849957) -(729850230--729850367) -(729850633--729850741) -(729940928--729940943) -(729940976--729940991) -(729942848--729942911) -(729944096--729944223) -(729974971--729975039) -(729975127--729975311) -(729975488--729975551) -(729977344--729977471) -(729977600--729977603) -(729978048--729978111) -(729
979520--729979647) -(729980544--729980671) -(730007552--730007615) -(730007808--730007935) -(730008108--730008111) -(730009648--730009775) -(730009979--730010031) -(730035456--730035647) -(730035712--730035775) -(730069248--730069375) -(730069440--730069503) -(730069568--730069631) -(730069696--730069823) -(730069952--730070079) -(730070144--730070207) -(730070336--730070463) -(730070528--730070655) -(730137664--730137727) -(730137792--730137919) -(730138672--730138783) -(730139072--730139455) -(730139584--730139647) -(730141568--730141655) -(730141672--730141759) -(730141888--730142207) -(730147456--730147583) -(730174953--730175080) -(730401536--730401727) -(730401792--730401855) -(730403038--730403101) -(730412843--730412927) -(730412992--730413055) -(730430400--730430527) -(730430720 --730430783) -(730437984--730438111) -(730438240--730438495) -(760710592--760710655) -(760716444--760716571) -(760716636--760716955) -(760718236--760718299) -(760720064--760720127) -(760746737--760746992) -(760777361--760777464) -(760777529--760777656) -(760777785--760777848) -(760777913--760777976) -(760778240--760778303) -(760778321--760778344) -(761182064--761182191) -(761196928--761197055) -(761197184--761197191) -(761197312--761197439) -(761197504--761197631) -(761197760--761197887) -(761198080--761198095) -(761198144--761198335) -(761198400--761198463) -(761198592--761198655) -(761199104--761199295) -(761199424--761199487) -(761243346--761243537) -(761428600--761428671) -(761428736--761428927) -(761428992--761429055) -(761429184--761429247) -(761429312--761429375) -(761429440--7614 29503) -(761457113--761457240) -(761457305--761457368) -(761457433--761457496) -(761459136--761459263) -(761459328--761459583) -(761459648--761459711) -(761460736--761460799) -(786337728--786337791) -(786342592--786342783) -(786347776--786347903) -(786350400--786350591) -(786351616--786351679) -(786400960--786401023) -(786416384--786416511) -(786416576--786416639) -(786416704--786416767) -(786535896--786535935) -(786860416--786860543) -(786860608--786860735) -(786860864--786860927) -(786861184--786861311) -(786891456--786891519) -(786906368--786906623) -(786906752--786906815) -(786906880--786906943) -(786907008--786907135) -(786915136--786915263) -(786915328--786915455) -(786915520--786915711) -(786915776--786915839) -(786916263--786916351) -(786927852--786927979) -(786928044--786928171)
-(786928300--786928427) -(786928620--786928747) -(786928812--786929003) -(786929772--786929899) -(786929964--786930091) -(786930156--786930175) -(786931264--786931455) -(786957184--786957311) -(786959237--786959300) -(786961176--786961239) -(786961368--786961559) -(786964672--786964735) -(786978176--786978303) -(786978686--786978687) -(786978752--786978815) -(787022400--787022591) -(787039261--787039295) -(787039360--787039423) -(787045152--787045183) -(787045312--787045375) -(787045440--787045503) -(787045568--787045695) -(787045888--787046143) -(787058688--787058815) -(787058880--787059007) -(787059136--787059263) -(787059328--787059455) -(787059584--787059711) -(787059776--787059839) -(787059968--787060095) -(787060160--787060223) -(787060288--787060351) -(787060416--787060543) -(787
074432--787074559) -(787074624--787074751) -(787088064--787088127) -(787091968--787092095) -(787248320--787248447) -(787252224--787252287) -(787273099--787273279) -(787273600--787273727) -(787286080--787286207) -(787286784--787286911) -(787286976--787287039) -(787287872--787287999) -(787288090--787288217) -(787288410--787288537) -(787308672--787308735) -(787317248--787317439) -(787317657--787317784) -(787317849--787317912) -(787319178--787319241) -(787319478--787319541) -(787320522--787320777) -(787320842--787320905) -(787320978--787321105) -(787325760--787325887) -(787348224--787348351) -(787348416--787348479) -(787348992--787349055) -787350783 -(787351232--787351295) -(787351358--787351421) -(787352464--787352655) -(787352768--787352831) -(787354850--787355041) -(787365504--787365695) -(787365760--787365823) -787415887 -(787416527--787416590) -(787417323--787417450) -(787418304--787418559) -(787418619--787418623) -(787419081--787419130) -(787421204--787421523) -(787424960--787425151) -(787425792--787425819) -(787431040--787431231) -(787431296--787431359) -(787449280--787449343) -(787449536--787449599) -(787450048--787450111) -(787450209--787450400) -(787451979--787452170) -(787461137--787461328) -(787461393--787461456) -(787481664--787481727) -(787481984--787482111) -(787482209--787482336) -(787482401--787482464) -(787482785--787482848) -(787483233--787483360) -(787483456--787483583) -(787484224--787484415) -(787484481--787484544) -(787484993--787485120) -(787509504--787509695) -(787509824--787509887) -787513953 -(787514048--787514111) -(787524313--787524440) -(787552 832--787553023) -(787553088--787553215) -(787553408--787553599) -(787553664--787553727) -787579746 -(787579776--787580031) -(787580040--787580223) -(787581056--787581127) -(787581248--787581311) -(787581780--787581803) -(787583040--787583167) -(787583232--787583423) -(787584160--787584223) -(787584384--787584511) -(787584576--787584703) -(787586291--787586330) -(787645184--787645375) -(787645440--787645503) -(787646976--787647103) -(787673088--787673215) -(787692011--787692031) -(787692096--787692159) -(787692224--787692287) -(787781781--787781844) -(787782229--787782356) -(787810368--787810494) -(787814464--787814591) -(787814732--787814739) -(787814856--787814967) -(787815232--787815295) -(787815424--787815487) -(787820800--787820991) -(787821056--787821183) -(787821248--787821311) -(7 87821376--787821503) -(787821696--787821759) -(787821824--787821951) -(787822144--787822207) -(787836282--787836409) -(787836912--787836985) -(787837050--787837177) -(787837306--787837369) -(787837434--787837497) -(787837562--787837689) -(787837882--787838009) -(787838074--787838137) -(787838266--787838329) -(787842496--787842623) -(787842816--787843071) -(787849216--787849343) -(787850048--787850111) -(787850240--787850303) -(787850368--787850431) -(787851072--787851199) -(787853237--787853300) -(787855616--787855679) -(787870272--787870463) -(787870592--787870719) -(787870848--787871039) -(787871232--787871359) -(787871552--787871615) -(787876356--787876611) -(787876676--787876803) -(787876996--787877059) -(787883392--787883583) -(787903296--787903487) -(787903552--787903615) -(7879075 20--787907583) -(787908096--787908159) -(787908948--787909011) -(787909780--787909907) -(787910064--787910199) -(787910208--787910263) -(787910320--787910511) -(787910576--787910703) -(787915776--787915967) -(787916032--787916223) -(787916352--787916415) -(787916480--787916543) -(787929600--787929663) -(787929984--787930175) -(787930304--787930367) -(787930432--787930495) -(787930560--787930687) -(787930752--787930815) -(787930944--787931135) -(787942121--787942207) -(787942720--787942911) -(787943333--787943396) -(787950080--787950207) -(787950272--787950335) -(787950400--787950527) -(787963200--787963327) -(787963392--787963583) -(787963712--787963775) -(787979038--787979101) -(788006830--788007021) -(788014784--788014911) -(788014976--788015103) -(788039680--788039743) -(788040000--78 8040127) -(788045056--788045183) -(788045798--788045861) -(788048960--788049215) -(788049408--788049535) -(788049664--788049791) -(788075840--788075903) -(788076251--788076442) -(788081600--788081663) -(788137408--788137471) -(788139247--788139519) -(788140800--788140863) -(788140903--788141030) -(788141159--788141222) -(788243776--788243903) -(788243968--788244159) -(788244352--788244543) -(788244608--788244671) -(788244864--788244927) -(788309112--788309119) -(788370432--788370623) -(788370688--788370815) -(788387904--788387967) -(788399936--788399999) -(788407936--788408127) -(788411392--788411519) -(788497600--788497663) -(788508102--788508165) -(788523200--788523455) -(788523520--788523647) -(788523712--788523775) -(788524032--788524223) -(788524352--788524479) -(788524544--78852460
7) -(788524672--788524735) -(788562824--788562943) -(788564076--788564203) -(788565888--788566015) -(788566464--788566527) -(788566657--788566727) -(788568000--788568063) -(788596096--788596223) -(788597071--788597134) -(788597896--788597966) -(788598031--788598158) -(788638536--788638783) -(788638792--788638855) -(788732480--788732799) -(788732928--788733055) -(788740608--788740735) -(788741440--788741503) -(788741568--788741695) -(788741760--788741887) -(788759424--788759487) -(788763672--788763735) -(788764096--788764159) -(788766596--788766659) -(788766928--788766935) -(788771028--788771155) -(788771220--788771411) -(788771604--788771731) -(788926567--788926694) -(789026048--789026175) -(789026368--789026495) -(789026560--789026751) -(789026880--789027007) -(789059007--789059134) -(7 89059199--789059274) -(789059287--789059338) -(789059391--789059518) -789119978 -(789256512--789256575) -(789256697--789256824) -(789480832--789480959) -(789481600--789481983) -(789486202--789486393) -(789486458--789486521) -(789486650--789486713) -(789493568--789493759) -(789513211--789513215) -(789516608--789516799) -(808519894--808519903) -(808520128--808520191) -808522431 -(808548068--808548095) -(808586123--808586143) -(808586161--808586168) -(809995722--809995775) -(810192542--810192559) -(810192860--810192917) -(810193814--810194005) -(810194070--810194197) -(810223833--810223871) -(810228160--810228223) -(810230245--810230271) -(810600384--810600447) -(810600983--810600991) -(810601010--810601023) -(811142068--811142079) -(811142120--811142143) -(811502667--811502719) -(811504311 --811504319) -(811504348--811504374) -(811504764--811504767) -(811994576--811994623) -(811995258--811995303) -(811995368--811995391) -(811996672--811996775) -(813437760--813437823) -(813697398--813697407) -(813697421--813697423) -(813698098--813698111) -(813698146--813698175) -(813698213--813698255) -(813698576--813698771) -(817341312--817341375) -(819337801--819338012) -(819338077--819338239) -(819339264--819339292) -(819430784--819430911) -(819431140--819431715) -(819431844--819431930) -(819432381--819432419) -(819469312--819469439) -(819469504--819469567) -(819469824--819469951) -(819531144--819531207) -(819531784--819531847) -(819536388--819536451) -(819562176--819562239) -(819562985--819563007) -(819563584--819563647) -(819563817--819563944) -(819564009--819564031) -(819579648--8195 79711) -(819597504--819597567) -(819598208--819598271) -(819598328--819598391) -(819598980--819599171) -(819602278--819602431) -(819726400--819726463) -(819726720--819726847) -(819726912--819726975) -(819733735--819733798) -(819733811--819733938) -(819737400--819737527) -(819737592--819737599) -(819758016--819758079) -(819758272--819758335) -(819761216--819761343) -(819825856--819825983) -(819901663--819901790) -(819901855--819901918) -(819901983--819902110) -(819902175--819902238) -(819902367--819902430) -(820019712--820019903) -(820019968--820020031) -(820021760--820021888) -(820022017--820022144) -(820022785--820022976) -(820023361--820023488) -(820029202--820029329) -(820029394--820029457) -(820029522--820029585) -(820029650--820029841) -(820030674--820030737) -(820053642--820053643) -(820053770--820053833) -(820057130--820057141) -(820059690--820059817) -(820059882--820059945) -(820060010--820060137) -(820061162--820061183) -(820086016--820086143) -(820086208--820086271) -(820087424--820087487) -(820087552--820087679) -(820088064--820088191) -(820089277--820089468) -(820089533--820089596) -(820091153--820091344) -(820091665--820091920) -(820282816--820283007) -(820283520--820283583) -(820284480--820284543) -(820284608--820284863) -(820288020--820288083) -(820290440--820290631) -(820290696--820290759) -(820291080--820291143) -(820342116--820342243) -(820342784--820342883) -(820342948--820343011) -(820415552--820415743) -(820415808--820415871) -(820415936--820415943) -(820445760--820445951) -(820448704--820448767) -(820449152--820449279) -(820455296--820455487) -(820
455616--820455743) -(820592923--820592959) -(820593024--820593151) -(820596928--820597055) -(820597120--820597183) -(820678208--820678399)
Fix? yes

Free blocks count wrong for group #11 (19451, counted=32754). Fix? yes

Free blocks count wrong for group #12 (3901, counted=31393). Fix? yes

Free blocks count wrong for group #13 (21626, counted=32624). Fix? yes

Free blocks count wrong for group #28 (28326, counted=28390). Fix? yes

Free blocks count wrong for group #3984 (3305, counted=3306). Fix? yes

Free blocks count wrong for group #6206 (32767, counted=32768). Fix? yes

Free blocks count wrong for group #10802 (4520, counted=4776). Fix? yes

Free blocks count wrong for group #10805 (4558, counted=4878). Fix? yes

Free blocks count wrong for group #11054 (2789, counted=2853). Fix? yes

Free blocks count wrong for group #12600 (2302, counted=2430). Fix? yes

Free blocks count wrong for group #12941 (2097, counted=2161). Fix? yes

Free blocks count wrong for group #13139 (2209, counted=2669). Fix? yes

Free blocks count wrong for group #22192 (12576, counted=13339). Fix? yes

Free blocks count wrong for group #22193 (13738, counted=14123). Fix? yes

Free blocks count wrong for group #22194 (15219, counted=15795). Fix? yes

Free blocks count wrong for group #22195 (16444, counted=17852). Fix? yes

Free blocks count wrong for group #22198 (12309, counted=12885). Fix? yes

Free blocks count wrong for group #22199 (11087, counted=11919). Fix? yes

Free blocks count wrong for group #22207 (29767, counted=29959). Fix? yes

Free blocks count wrong for group #22210 (29191, counted=29255). Fix? yes

Free blocks count wrong for group #22248 (7671, counted=8119). Fix? yes

Free blocks count wrong for group #22250 (8384, counted=8457). Fix? yes

Free blocks count wrong for group #22252 (5498, counted=5614). Fix? yes

Free blocks count wrong for group #22253 (5950, counted=6424). Fix? yes

Free blocks count wrong for group #22254 (5754, counted=5762). Fix? yes

Free blocks count wrong for group #22255 (2735, counted=2762). Fix? yes

Free blocks count wrong for group #22256 (384, counted=1344). Fix? yes

Free blocks count wrong for group #22262 (4517, counted=4709). Fix? yes

Free blocks count wrong for group #22265 (10172, counted=10364). Fix? yes

Free blocks count wrong for group #22269 (11089, counted=11281). Fix? yes

Free blocks count wrong for group #22270 (11819, counted=11919). Fix? yes

Free blocks count wrong for group #22271 (8072, counted=8392). Fix? yes

Free blocks count wrong for group #22273 (20376, counted=21007). Fix? yes

Free blocks count wrong for group #22276 (17171, counted=17395). Fix? yes

Free blocks count wrong for group #22277 (15748, counted=16518). Fix? yes

Free blocks count wrong for group #22278 (15052, counted=15685). Fix? yes

Free blocks count wrong for group #22279 (18977, counted=19809). Fix? yes

Free blocks count wrong for group #22282 (16928, counted=18304). Fix? yes

Free blocks count wrong for group #22283 (20180, counted=20308). Fix? yes

Free blocks count wrong for group #22290 (21832, counted=22493). Fix? yes

Free blocks count wrong for group #22291 (20172, counted=20556). Fix? yes

Free blocks count wrong for group #23215 (8431, counted=9071). Fix? yes

Free blocks count wrong for group #23216 (6597, counted=6853). Fix? yes

Free blocks count wrong for group #23217 (9092, counted=9540). Fix? yes

Free blocks count wrong for group #23229 (11090, counted=12330). Fix? yes

Free blocks count wrong for group #23231 (4909, counted=5101). Fix? yes

Free blocks count wrong for group #23236 (7480, counted=8000). Fix? yes

Free blocks count wrong for group #23237 (6967, counted=7735). Fix? yes

Free blocks count wrong for group #23997 (20764, counted=21404). Fix? yes

Free blocks count wrong for group #23999 (8373, counted=8693). Fix? yes

Free blocks count wrong for group #24003 (12530, counted=12570). Fix? yes

Free blocks count wrong for group #24013 (12805, counted=13253). Fix? yes

Free blocks count wrong for group #24014 (14399, counted=15576). Fix? yes

Free blocks count wrong for group #24015 (17929, counted=19101). Fix? yes

Free blocks count wrong for group #24016 (15799, counted=16505). Fix? yes

Free blocks count wrong for group #24018 (17925, counted=18760). Fix? yes

Free blocks count wrong for group #24019 (12575, counted=13919). Fix? yes

Free blocks count wrong for group #24020 (12440, counted=12632). Fix? yes

Free blocks count wrong for group #24024 (15934, counted=16062). Fix? yes

Free blocks count wrong for group #24025 (18816, counted=19189). Fix? yes

Free blocks count wrong for group #24026 (17781, counted=18549). Fix? yes

Free blocks count wrong for group #24027 (14869, counted=16213). Fix? yes

Free blocks count wrong for group #24028 (16895, counted=17728). Fix? yes

Free blocks count wrong for group #24030 (14495, counted=15795). Fix? yes

Free blocks count wrong for group #24031 (12470, counted=13302). Fix? yes

Free blocks count wrong for group #24032 (14754, counted=16098). Fix? yes

Free blocks count wrong for group #24033 (14902, counted=15095). Fix? yes

Free blocks count wrong for group #24034 (3898, counted=4474). Fix? yes

Free blocks count wrong for group #24035 (3651, counted=4932). Fix? yes

Free blocks count wrong for group #24037 (6625, counted=7137). Fix? yes

Free blocks count wrong for group #24038 (19122, counted=19271). Fix? yes

Free blocks count wrong for group #24041 (13985, counted=14177). Fix? yes

Free blocks count wrong for group #24042 (10693, counted=12806). Fix? yes

Free blocks count wrong for group #24043 (17017, counted=18681). Fix? yes

Free blocks count wrong for group #24044 (14013, counted=14909). Fix? yes

Free blocks count wrong for group #24045 (14940, counted=17052). Fix? yes

Free blocks count wrong for group #24046 (15428, counted=16475). Fix? yes

Free blocks count wrong for group #24047 (17978, counted=18042). Fix? yes

Free blocks count wrong for group #24048 (14155, counted=14603). Fix? yes

Free blocks count wrong for group #24049 (13203, counted=14099). Fix? yes

Free blocks count wrong for group #24050 (16122, counted=16442). Fix? yes

Free blocks count wrong for group #24052 (12096, counted=12689). Fix? yes

Free blocks count wrong for group #24055 (17728, counted=18368). Fix? yes

Free blocks count wrong for group #24057 (12112, counted=12120). Fix? yes

Free blocks count wrong for group #24059 (15984, counted=16368). Fix? yes

Free blocks count wrong for group #24060 (16487, counted=16871). Fix? yes

Free blocks count wrong for group #24063 (21110, counted=22134). Fix? yes

Free blocks count wrong for group #24065 (11780, counted=12355). Fix? yes

Free blocks count wrong for group #24066 (12148, counted=12539). Fix? yes

Free blocks count wrong for group #24067 (14385, counted=14697). Fix? yes

Free blocks count wrong for group #24070 (22106, counted=23002). Fix? yes

Free blocks count wrong for group #24071 (18484, counted=19196). Fix? yes

Free blocks count wrong for group #24076 (14295, counted=14423). Fix? yes

Free blocks count wrong for group #24079 (10420, counted=10996). Fix? yes

Free blocks count wrong for group #24080 (17261, counted=17645). Fix? yes

Free blocks count wrong for group #24082 (20265, counted=20266). Fix? yes

Free blocks count wrong for group #24086 (13718, counted=13910). Fix? yes

Free blocks count wrong for group #24093 (14828, counted=15852). Fix? yes

Free blocks count wrong for group #24094 (11301, counted=11498). Fix? yes

Free blocks count wrong for group #24674 (8195, counted=8298). Fix? yes

Free blocks count wrong for group #24676 (13364, counted=13393). Fix? yes

Free blocks count wrong for group #24719 (14120, counted=14174). Fix? yes

Free blocks count wrong for group #24725 (14235, counted=14631). Fix? yes

Free blocks count wrong for group #24726 (13671, counted=13801). Fix? yes

Free blocks count wrong for group #24737 (12783, counted=12870). Fix? yes

Free blocks count wrong for group #24754 (20326, counted=20362). Fix? yes

Free blocks count wrong for group #24765 (16906, counted=16999). Fix? yes

Free blocks count wrong for group #24780 (19463, counted=19685). Fix? yes

Free blocks count wrong for group #24824 (16732, counted=16796). Fix? yes

Free blocks count wrong for group #24832 (9611, counted=9907). Fix? yes

Free blocks count wrong for group #24943 (8131, counted=8195). Fix? yes

Free blocks count wrong for group #25004 (9390, counted=9794). Fix? yes

Free blocks count wrong for group #25007 (14790, counted=15620). Fix? yes

Free blocks count wrong for group #25008 (10586, counted=10906). Fix? yes

Free blocks count wrong for group #25010 (11589, counted=11781). Fix? yes

Free blocks count wrong for group #25011 (10392, counted=10758). Fix? yes

Free blocks count wrong for group #25012 (9630, counted=10168). Fix? yes

Free blocks count wrong for group #25016 (11461, counted=12045). Fix? yes

Free blocks count wrong for group #25017 (11176, counted=11432). Fix? yes

Free blocks count wrong for group #25019 (14786, counted=14914). Fix? yes

Free blocks count wrong for group #25021 (7922, counted=8370). Fix? yes

Free blocks count wrong for group #25025 (9455, counted=10800). Fix? yes

Free blocks count wrong for group #25026 (9967, counted=10387). Fix? yes

Free blocks count wrong for group #25027 (7700, counted=8916). Fix? yes

Free blocks count wrong for group #25033 (12524, counted=13484). Fix? yes

Free blocks count wrong for group #25034 (7606, counted=7898). Fix? yes

Free blocks count wrong for group #25037 (7555, counted=7819). Fix? yes

Free blocks count wrong for group #25038 (5607, counted=6311). Fix? yes

Free blocks count wrong for group #25042 (5996, counted=6353). Fix? yes

Free blocks count wrong for group #25045 (6953, counted=7145). Fix? yes

Free blocks count wrong (573250164, counted=573366986). Fix? yes

Inode bitmap differences: -32641213
Fix? yes

Free inodes count wrong for group #3984 (2420, counted=2421). Fix? yes

Free inodes count wrong (361075851, counted=361075852). Fix? yes

/dev/vgstore/lvol0: ***** FILE SYSTEM WAS MODIFIED *****

1731444 inodes used (0.48%)
24100 non-contiguous files (1.4%)

    1147 non-contiguous directories (0.1%)
         # of inodes with ind/dind/tind blocks: 0/0/0
         Extent depth histogram: 1569321/12050/188

877841718 blocks used (60.49%)
0 bad blocks
75 large files

1440425 regular files
140400 directories
0 character device files
0 block device files
0 fifos
0 links
150610 symbolic links (149875 fast symbolic links)
0 sockets
--------
1731435 files

-- Andreas

-----Original Message-----
From: Eric Sandeen <[email protected]>
To: Andreas Friedrich Berendsen <[email protected]>
Cc: linux-ext4 <[email protected]>
Subject: Re: kernel BUG at fs/ext4/extents.c:2738!
Date: Fri, 20 Feb 2009 14:40:31 -0600

please stop w/ the file removal for a moment, and see if you can make an e2image of the filesystem which reproduces the e2fsck problem ....

-Eric

__________________________________________
Andreas Friedrich Berendsen
SCA OCP MSCA A+ Linux+ Network+ HpMASE

Have you tried running e2fsck? It sounds like a filesystem corruption is triggering a BUG() message, which is obviously a bug in and of itself. :-)

- Ted

Tso,

I used fsck many times but an error as identified at a certain block and fsck was aborting with a Segment fault error. I had to fix all the errors manually (not using -p or -y), skipping this specific problem.

After that I used reszie2fs -pM for just the first step. Mounting the filesystem, removing many files (which triggered the BUG), I was able to run fsck with -y many times.

Anyway, the problem seems to be under control but I'm not sure. The new kernel (2.6.28.7) has a long list of ext4 errors corrected and I'm compiling it right now. Let's see what happens next :)

Thanks anyway

-- Andreas

-----Original Message-----
From: Theodore Tso <[email protected]>
To: Andreas Friedrich Berendsen <[email protected]>
Cc: linux-ext4 <[email protected]>
Bcc: [email protected]
Subject: Re: kernel BUG at fs/ext4/extents.c:2738!
Date: Fri, 20 Feb 2009 14:30:13 -0500

Have you tried running e2fsck? It sounds like a filesystem corruption is triggering a BUG() message, which is obviously a bug in and of itself. :-)

- Ted

__________________________________________
Andreas Friedrich Berendsen
SCA OCP MSCA A+ Linux+ Network+ HpMASE

What version of e2fsck were you using, and what was the failure? *Please* don't ignore stuff like this. Report it, and mention it when you report kernel problems. It really helps get to the bottom of things.

I'm going to guess you were using something older than e2fsprogs 1.41.4, and you were running into the bug which was fixed in this e2fsprogs commit.

commit 7518c176867099eb529502103106501861a71280

Author: Theodore Ts'o <[email protected]>
Date:   Thu Dec 25 22:42:38 2008 -0500

e2fsck: Fix an unhandled corruption case in scan_extent_node()

A corrupted interior node in an extent tree would cause e2fsck to crash with the error message:

Error1: Corrupt extent header on inode 107192
Aborted (core dumped)

Handle this and related failures when scanning an inode's extent tree more robustly.

A corrupted extent could very easily have explained the symptoms you reported. (No, the kernel still shouldn't have flagged a BUG(), but the fact of the matter is the filesystme was corrupted.)

I'd recommend making sure you're running the latest version of e2fsprogs and use e2fsck to make sure the filesystem is fully consistent.

If e2fsck ever core dumps, please report it as a bug. The e2fsck man page has a section, REPORTING BUGS, that goes into details about how to send a useful bug report about an e2fsck failure.

- Ted

Tso,

If you look my previous e-mails, you will see that I included an output fro my fsck program, which is version 1.41.4 (second output bewlo):

# fsck
fsck 1.41.4 (27-Jan-2009)
e2fsck 1.41.4 (27-Jan-2009)

My first action when the problem happened was to check the efsprog page, new kernel version (chancgelog), Google (of course), and friends. After all those resources where exausted I decided to send a message to this e-mail group.

Right now, after running efsck.ext4 program many times over that filesystem, seems that no corruptions where identified.

-- Andreas

-----Original Message-----
From: Theodore Tso <[email protected]>
To: Andreas Friedrich Berendsen <[email protected]>
Cc: linux-ext4 <[email protected]>
Bcc: [email protected]
Subject: Re: kernel BUG at fs/ext4/extents.c:2738!
Date: Sat, 21 Feb 2009 23:31:26 -0500

What version of e2fsck were you using, and what was the failure? *Please* don't ignore stuff like this. Report it, and mention it when you report kernel problems. It really helps get to the bottom of things.

I'm going to guess you were using something older than e2fsprogs 1.41.4, and you were running into the bug which was fixed in this e2fsprogs commit.

commit 7518c176867099eb529502103106501861a71280

Author: Theodore Ts'o <[email protected]>
Date:   Thu Dec 25 22:42:38 2008 -0500

e2fsck: Fix an unhandled corruption case in scan_extent_node()

A corrupted interior node in an extent tree would cause e2fsck to crash with the error message:

Error1: Corrupt extent header on inode 107192
Aborted (core dumped)

Handle this and related failures when scanning an inode's extent tree more robustly.

A corrupted extent could very easily have explained the symptoms you reported. (No, the kernel still shouldn't have flagged a BUG(), but the fact of the matter is the filesystme was corrupted.)

I'd recommend making sure you're running the latest version of e2fsprogs and use e2fsck to make sure the filesystem is fully consistent.

If e2fsck ever core dumps, please report it as a bug. The e2fsck man page has a section, REPORTING BUGS, that goes into details about how to send a useful bug report about an e2fsck failure.

- Ted

__________________________________________
Andreas Friedrich Berendsen
SCA OCP MSCA A+ Linux+ Network+ HpMASE

So this was the version where e2fsck core dumped? Do you remember what sort of messages it printed out before the core dump? That would have been very useful indeed.

- Ted