2 year old raid1 issue chunk-recovery help

DISCUSSION Vladi Gergov 2 participants 6 messages Jan 15, 2014 – Jan 20, 2014 Btrfs file system

Share & tools

Vladi Gergov

author

2014-01-15 19:48 UTC #0 lore raw

Hi, in 2010 i had an issue with my raid1 when one drive failed and i added another drive to the array and tried to rebuild. Here is what bug I hit according to Chris Mason
http://www.mail-archive.com/[email protected]/msg06868.html

I have since updated to lastes btrfs-tools 3.12 + kernel 3.13-rc7 and attempted an chunk recovery which failed with this
http://bpaste.net/show/168445/

If anyone can help me get at least some of the data off this bad boy it would be great! I am cc'ing Miao since his name was thrown under the bus in irc :). Thanks in advance!

Signature · 2 lines

,-| Vladi
`-| Gergov

Miao Xie

2014-01-16 02:09 UTC #1 lore raw

Quoted 5 lines · Vladi Gergov

Hi, in 2010 i had an issue with my raid1 when one drive failed and i added another drive to the array and tried to rebuild. Here is what bug I hit according to Chris Mason
http://www.mail-archive.com/[email protected]/msg06868.html

Quoted 8 lines

I have since updated to lastes btrfs-tools 3.12 + kernel 3.13-rc7 and attempted an chunk recovery which failed with this
http://bpaste.net/show/168445/
If anyone can help me get at least some of the data off this bad boy it would be great! I am cc'ing Miao since his name was thrown under the bus in irc :). Thanks in advance!

Chunk recover command can only recover the case that the devices are good, only the chunk tree is corrupted. So it is not suitable to fix your issue.

I think you can try the replace function if you can mount the device successfully, just like:

 # mount <dev> -o degraded <mnt>
 # btrfs replace start missing <new_dev>

Thanks
Miao

Vladi Gergov

author

2014-01-16 18:23 UTC #2 lore raw

Thanks Miao,

I have tried to mount it with -o degraded and -o recovery here is the outputs:

[216094.269443] btrfs: device label das4 devid 2 transid 107954
/dev/loop7
[216094.281965] btrfs: device label das1 devid 7 transid 1168964
/dev/sdi
[216094.313419] btrfs: device label das4 devid 3 transid 107954 /dev/sdj
[216113.887503] btrfs: device label das4 devid 2 transid 107954
/dev/loop7
[216113.888690] btrfs: allowing degraded mounts
[216113.889440] btrfs: failed to read chunk root on loop7
[216113.905742] btrfs: open_ctree failed
[216135.144739] btrfs: device label das4 devid 2 transid 107954
/dev/loop7
[216135.145996] btrfs: enabling auto recovery
[216135.146783] btrfs: failed to read chunk root on loop7
[216135.155985] btrfs: open_ctree failed

any other suggestions? Thanks again.

Quoted 26 lines · Miao Xie

Quoted 5 lines · Vladi Gergov
Hi, in 2010 i had an issue with my raid1 when one drive failed and i added another drive to the array and tried to rebuild. Here is what bug I hit according to Chris Mason
http://www.mail-archive.com/[email protected]/msg06868.html
Quoted 8 lines
I have since updated to lastes btrfs-tools 3.12 + kernel 3.13-rc7 and attempted an chunk recovery which failed with this
http://bpaste.net/show/168445/
If anyone can help me get at least some of the data off this bad boy it would be great! I am cc'ing Miao since his name was thrown under the bus in irc :). Thanks in advance!
Chunk recover command can only recover the case that the devices are good, only the chunk tree is corrupted. So it is not suitable to fix your issue.
I think you can try the replace function if you can mount the device successfully, just like:
 # mount <dev> -o degraded <mnt>
 # btrfs replace start missing <new_dev>
Thanks
Miao

Signature · 2 lines

,-| Vladi
`-| Gergov

Miao Xie

2014-01-17 02:25 UTC #3 lore raw

Quoted 21 lines · Vladi Gergov

Thanks Miao,

I have tried to mount it with -o degraded and -o recovery here is the outputs:

[216094.269443] btrfs: device label das4 devid 2 transid 107954
/dev/loop7
[216094.281965] btrfs: device label das1 devid 7 transid 1168964
/dev/sdi
[216094.313419] btrfs: device label das4 devid 3 transid 107954 /dev/sdj
[216113.887503] btrfs: device label das4 devid 2 transid 107954
/dev/loop7
[216113.888690] btrfs: allowing degraded mounts
[216113.889440] btrfs: failed to read chunk root on loop7
[216113.905742] btrfs: open_ctree failed
[216135.144739] btrfs: device label das4 devid 2 transid 107954
/dev/loop7
[216135.145996] btrfs: enabling auto recovery
[216135.146783] btrfs: failed to read chunk root on loop7
[216135.155985] btrfs: open_ctree failed

Quoted 1 line

any other suggestions? Thanks again.

Is loop7 used to instead of the bad device /dev/sdi? If so, I think we should erase the data in the loop7, and then

Quoted 2 lines

 # mount <dev> -o degraded <mnt>
 # btrfs replace start missing <new_dev>

Thanks
Miao

Quoted 29 lines · Miao Xie

Quoted 26 lines · Vladi Gergov
Quoted 5 lines
Hi, in 2010 i had an issue with my raid1 when one drive failed and i added another drive to the array and tried to rebuild. Here is what bug I hit according to Chris Mason
http://www.mail-archive.com/[email protected]/msg06868.html
Quoted 8 lines
I have since updated to lastes btrfs-tools 3.12 + kernel 3.13-rc7 and attempted an chunk recovery which failed with this
http://bpaste.net/show/168445/
If anyone can help me get at least some of the data off this bad boy it would be great! I am cc'ing Miao since his name was thrown under the bus in irc :). Thanks in advance!
Chunk recover command can only recover the case that the devices are good, only the chunk tree is corrupted. So it is not suitable to fix your issue.
I think you can try the replace function if you can mount the device successfully, just like:
 # mount <dev> -o degraded <mnt>
 # btrfs replace start missing <new_dev>
Thanks
Miao

Vladi Gergov

author

2014-01-17 16:55 UTC #4 lore raw

Hi,

Not sure if my previous email was received as I sent it from my phone. I had to dd the disk off and then losetup mount the image. What do you mean by erase the data on loop7? I have tried to mount separately without success.

Quoted 63 lines · Miao Xie

Quoted 21 lines · Vladi Gergov
Thanks Miao,
I have tried to mount it with -o degraded and -o recovery here is the outputs:
[216094.269443] btrfs: device label das4 devid 2 transid 107954
/dev/loop7
[216094.281965] btrfs: device label das1 devid 7 transid 1168964
/dev/sdi
[216094.313419] btrfs: device label das4 devid 3 transid 107954 /dev/sdj
[216113.887503] btrfs: device label das4 devid 2 transid 107954
/dev/loop7
[216113.888690] btrfs: allowing degraded mounts
[216113.889440] btrfs: failed to read chunk root on loop7
[216113.905742] btrfs: open_ctree failed
[216135.144739] btrfs: device label das4 devid 2 transid 107954
/dev/loop7
[216135.145996] btrfs: enabling auto recovery
[216135.146783] btrfs: failed to read chunk root on loop7
[216135.155985] btrfs: open_ctree failed
Quoted 1 line
any other suggestions? Thanks again.
Is loop7 used to instead of the bad device /dev/sdi? If so, I think we should erase the data in the loop7, and then
Quoted 2 lines
Quoted 2 lines
 # mount <dev> -o degraded <mnt>
 # btrfs replace start missing <new_dev>
Thanks
Miao
Quoted 29 lines · Miao Xie
Quoted 26 lines · Vladi Gergov
Quoted 5 lines
Hi, in 2010 i had an issue with my raid1 when one drive failed and i added another drive to the array and tried to rebuild. Here is what bug I hit according to Chris Mason
http://www.mail-archive.com/[email protected]/msg06868.html
Quoted 8 lines
I have since updated to lastes btrfs-tools 3.12 + kernel 3.13-rc7 and attempted an chunk recovery which failed with this
http://bpaste.net/show/168445/
If anyone can help me get at least some of the data off this bad boy it would be great! I am cc'ing Miao since his name was thrown under the bus in irc :). Thanks in advance!
Chunk recover command can only recover the case that the devices are good, only the chunk tree is corrupted. So it is not suitable to fix your issue.
I think you can try the replace function if you can mount the device successfully, just like:
 # mount <dev> -o degraded <mnt>
 # btrfs replace start missing <new_dev>
Thanks
Miao

Signature · 2 lines

,-| Vladi
`-| Gergov

Miao Xie

2014-01-20 02:30 UTC #5 lore raw

Quoted 4 lines · Vladi Gergov

Not sure if my previous email was received as I sent it from my phone. I had to dd the disk off and then losetup mount the image. What do you mean by erase the data on loop7? I have tried to mount separately without success.

When we mount a btrfs filesystem, the kernel will find out all the devices that belongs to the filesystem by searching all the devices and comparing fs uuid in the super block. So I think you can erase the super block data in loop7 to prevent it being found since it broken the mount. But I made a mistake, actually you needn't erase the data on loop7, just detach it.

BTW, I have tried the method I said below on my box, and the device was replaced successfully.

 # mkfs.btrfs -d raid1 -m raid1 <dev1> <dev2>
 # mount <dev1> <mnt>
 # dd if=<in> of=<mnt>/<out> bs=1M count=1024
 # umount <mnt>
 # unplug <dev2>
 # mount <dev1> -o degraded <mnt>
 # btrfs replace start missing <new_dev>

Thanks
Miao

Quoted 66 lines · Miao Xie

Quoted 63 lines · Vladi Gergov
Quoted 21 lines
Thanks Miao,
I have tried to mount it with -o degraded and -o recovery here is the outputs:
[216094.269443] btrfs: device label das4 devid 2 transid 107954
/dev/loop7
[216094.281965] btrfs: device label das1 devid 7 transid 1168964
/dev/sdi
[216094.313419] btrfs: device label das4 devid 3 transid 107954 /dev/sdj
[216113.887503] btrfs: device label das4 devid 2 transid 107954
/dev/loop7
[216113.888690] btrfs: allowing degraded mounts
[216113.889440] btrfs: failed to read chunk root on loop7
[216113.905742] btrfs: open_ctree failed
[216135.144739] btrfs: device label das4 devid 2 transid 107954
/dev/loop7
[216135.145996] btrfs: enabling auto recovery
[216135.146783] btrfs: failed to read chunk root on loop7
[216135.155985] btrfs: open_ctree failed
Quoted 1 line
any other suggestions? Thanks again.
Is loop7 used to instead of the bad device /dev/sdi? If so, I think we should erase the data in the loop7, and then
Quoted 2 lines
Quoted 2 lines
 # mount <dev> -o degraded <mnt>
 # btrfs replace start missing <new_dev>
Thanks
Miao
Quoted 29 lines · Miao Xie
Quoted 26 lines · Vladi Gergov
Quoted 5 lines
Hi, in 2010 i had an issue with my raid1 when one drive failed and i added another drive to the array and tried to rebuild. Here is what bug I hit according to Chris Mason
http://www.mail-archive.com/[email protected]/msg06868.html
Quoted 8 lines
I have since updated to lastes btrfs-tools 3.12 + kernel 3.13-rc7 and attempted an chunk recovery which failed with this
http://bpaste.net/show/168445/
If anyone can help me get at least some of the data off this bad boy it would be great! I am cc'ing Miao since his name was thrown under the bus in irc :). Thanks in advance!
Chunk recover command can only recover the case that the devices are good, only the chunk tree is corrupted. So it is not suitable to fix your issue.
I think you can try the replace function if you can mount the device successfully, just like:
 # mount <dev> -o degraded <mnt>
 # btrfs replace start missing <new_dev>
Thanks
Miao