Skip to contentSkip to conversation

This time HD LED doesn't come up at all :-(

DISCUSSION Swâmi Petaramesh 2 participants 6 messages Mar 28, 2013 Btrfs file system

lore
Share & tools
lore
#0 lore raw

(Resending without the attachment which didn't seem to make it thru...)

Hi folks,

This time, at reboot, one of my machines won't come up.

Starting it in "single and verbose" mode shows that BTRFS looks dead (or at least seriously wounded) :

Kernel seems to mount BTRFS, then swap, then sits there doing nothing for quite a while (HD LED is OFF).

Then it starts spitting ugly messages about init being blocked for more than 120 seconds, btrfs-cleaner (the same) etc. - please see attached screenshot.

Any clue for getting out of this annoying situation without resorting to reinstall from backups ?

TIA.

Signature · 1 line
Swâmi Petaramesh <[email protected]> http://petaramesh.org PGP 9076E32E
#1 lore raw
Quoted 12 lines · Swâmi Petaramesh

(Resending without the attachment which didn't seem to make it thru...)

Hi folks,

This time, at reboot, one of my machines won't come up.

Starting it in "single and verbose" mode shows that BTRFS looks dead (or at least seriously wounded) :

Kernel seems to mount BTRFS, then swap, then sits there doing nothing for quite a while (HD LED is OFF).

Quoted 7 lines

Then it starts spitting ugly messages about init being blocked for more than 120 seconds, btrfs-cleaner (the same) etc. - please see attached screenshot.

Any clue for getting out of this annoying situation without resorting to reinstall from backups ?

Can you capture the output and post it somewhere so I can see it? Thanks,

Josef

#2 lore raw
Quoted 1 line · Josef Bacik

Can you capture the output and post it somewhere so I can see it? Thanks,

Hi Josef,

I just uploaded the screenshot here: http://dl.free.fr/pRabNdCsf (You will need to type in your email address or the list's one to retrieve it)

Meanwhile I did some more tests, and I'm puzzled :

- Booting the latest PartedMagic Live (I believe it's a 3.7.9 kernel), I can mount the HD BTRFS filesystem wihtout problem (only tried with subvolid=0), I can btrfsck it (unmounted), and it looks allright;

- Still, trying to boot the system the normal way (3.8.0 kernel) fails, and I get these errors...

At least the FS doesn't look plain dead, but I have no clue...

Kind regards.

Signature · 2 lines
Swâmi Petaramesh <[email protected]> http://petaramesh.org PGP 9076E32E
Ne cherchez pas : Je ne suis pas sur Facebook.
#3 lore raw
Quoted 9 lines · Swâmi Petaramesh
Quoted 1 line · Josef Bacik

Can you capture the output and post it somewhere so I can see it? Thanks,

Hi Josef,

I just uploaded the screenshot here: http://dl.free.fr/pRabNdCsf (You will need to type in your email address or the list's one to retrieve it)

Meanwhile I did some more tests, and I'm puzzled :

Quoted 9 lines

- Booting the latest PartedMagic Live (I believe it's a 3.7.9 kernel), I can mount the HD BTRFS filesystem wihtout problem (only tried with subvolid=0), I can btrfsck it (unmounted), and it looks allright;

- Still, trying to boot the system the normal way (3.8.0 kernel) fails, and I get these errors...

At least the FS doesn't look plain dead, but I have no clue...

So can you try a livecd thats 3.8 based and mount manually, and when it hangs get sysrq+w? The other option is to mount without -o inode_cache and see if that helps? It looks like you may be hitting some weird deadlock with inode_cache, but I can't say for sure without sysrq+w so I can see what the caching thread is doing. Thanks,

Josef

#4 lore raw
In reply to Swâmi Petaramesh
Quoted 9 lines · Swâmi Petaramesh
Quoted 1 line · Josef Bacik

Can you capture the output and post it somewhere so I can see it? Thanks,

Hi Josef,

I just uploaded the screenshot here: http://dl.free.fr/pRabNdCsf (You will need to type in your email address or the list's one to retrieve it)

Meanwhile I did some more tests, and I'm puzzled :

Quoted 9 lines

- Booting the latest PartedMagic Live (I believe it's a 3.7.9 kernel), I can mount the HD BTRFS filesystem wihtout problem (only tried with subvolid=0), I can btrfsck it (unmounted), and it looks allright;

- Still, trying to boot the system the normal way (3.8.0 kernel) fails, and I get these errors...

At least the FS doesn't look plain dead, but I have no clue...

Actually I see the deadlock now, sorry, I'll fix it up and send a patch. Thanks,

Josef

#5 lore raw
Quoted 1 line · Josef Bacik

Actually I see the deadlock now, sorry, I'll fix it up and send a patch.

Meanwhile, I booted in PartedMagic again (My Hero !), mounted the FS, chrooted into it, mounted everything under it, then I rebuilt the initramfs, unmounted and restarted "the usual way".

Then the HD LED stayed LIT for at least 5 full minutes, and in the end it would boot properly.

I don't know what exactly happened, but it seems that regenerating the initramfs from within the live CD somehow helped BTRFS to fix itself ?

Happily enough for me, for that's my wife's system, and you know bad things can happen when wife is unhappy :-\

(She's actually already badly unhappy about performance, leaves the machine "suspended" all the time for she can't stand the time it takes to boot, but if I tell her that her Linux FS died, I'm afraid she's back to WinXP in minutes :-/ which is worse than death...)

Thanks for your help :-)

FYI parts of this machine's fstab and mountinfo in case it might help...

# grep btrfs fstab
/dev/sda1       /               btrfs  
subvol=@,compress=lzo,space_cache,inode_cache,autodefrag,relatime       0 0
/dev/sda1       /boot           btrfs  
subvol=@boot,space_cache,inode_cache,autodefrag,relatime                               
0 0
/dev/sda1       /tmp            btrfs  
subvol=@tmp,compress=lzo,space_cache,inode_cache,autodefrag,relatime    0 0
/dev/sda1       /home           btrfs  
subvol=@home,compress=lzo,space_cache,inode_cache,autodefrag,relatime   0 0
/dev/sda1       /storage/owner_.VBox  btrfs  
subvol=@owner_.VBox,compress=lzo,space_cache,inode_cache,autodefrag,noatime  
0 0
/dev/sda1       /storage/owner_BIG    btrfs  
subvol=@owner_BIG,compress=lzo,space_cache,inode_cache,autodefrag,noatime    
0 0
/dev/sda1       /mnt/ROOTFS     btrfs  
noauto,compress=lzo,space_cache,inode_cache,autodefrag,relatime 0 0
# grep btrfs /proc/self/mountinfo
20 1 0:16 /@ / rw,relatime - btrfs /dev/sda1
rw,compress=lzo,space_cache,autodefrag,inode_cache
27 20 0:16 /@tmp /tmp rw,relatime - btrfs /dev/sda1
rw,compress=lzo,space_cache,autodefrag,inode_cache
28 20 0:16 /@boot /boot rw,relatime - btrfs /dev/sda1
rw,compress=lzo,space_cache,autodefrag,inode_cache
29 20 0:16 /@home /home rw,relatime - btrfs /dev/sda1
rw,compress=lzo,space_cache,autodefrag,inode_cache
30 20 0:16 /@owner_.VBox /storage/owner_.VBox rw,noatime - btrfs
/dev/sda1 rw,compress=lzo,space_cache,autodefrag,inode_cache
31 20 0:16 /@owner_BIG /storage/owner_BIG rw,noatime - btrfs /dev/sda1
rw,compress=lzo,space_cache,autodefrag,inode_cache
Signature · 2 lines
Swâmi Petaramesh <[email protected]> http://petaramesh.org PGP 9076E32E
Ne cherchez pas : Je ne suis pas sur Facebook.