Skip to contentSkip to conversation

How To Recover Files From ext3 Partition??

DISCUSSION UZAIR LAKHANI 5 participants 10 messages May 4, 2006 – May 8, 2006 VFS

lore
Share & tools
lore
#0 lore raw

Hello All,

Is there any way to recover the deleted files from a folder in ext3 partition. Basically in one directory, I gave the command (rm -rf *) and all the files and folders of this directory are lost. How to recover these. I had once found an article regarding un-deleting ext2 files.

Thanks,
Uzair Lakhani

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

#1 lore raw
Quoted 6 lines · UZAIR LAKHANI

Is there any way to recover the deleted files from a folder in ext3 partition. Basically in one directory, I gave the command (rm -rf *) and all the files and folders of this directory are lost. How to recover these. I had once found an article regarding un-deleting ext2 files.

It *should* work the same, though the the last time I tried such a recovery from and ext3 filesystem, ext3 made it particularly hard cause it zeros most of the inode on delete (including ext3_inode->i_block[]). Ext2 leaves much more information behind.

You could try with the undelfs feature in Midnight Commander, but first be sure to mount the filesystem read-only in order to avoid any further changes.

Erik

Signature · 2 lines
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands
#2 lore raw
Quoted 20 lines · --- Erik Mouw
Quoted 1 line · UZAIR LAKHANI

Is there any way to recover the deleted files from

a

Quoted 1 line

folder in ext3 partition. Basically in one

directory,

Quoted 1 line

I gave the command (rm -rf *) and all the files

and

Quoted 3 lines

folders of this directory are lost. How to recover these. I had once found an article regarding un-deleting ext2 files.

It *should* work the same, though the the last time I tried such a
recovery from and ext3 filesystem, ext3 made it particularly hard cause
it zeros most of the inode on delete (including ext3_inode->i_block[]).
Ext2 leaves much more information behind.

Quoted 9 lines

You could try with the undelfs feature in Midnight Commander, but first
be sure to mount the filesystem read-only in order to avoid any further
changes.

Erik

Hello All,

Thanks for the reply. Yes it is difficult to recover files from ext3 as compared to ext2. I found this helpful info from one website.

Q: How can I recover (undelete) deleted files from my ext3 partition?
Actually, you can't! This is what one of the developers, Andreas Dilger, said about it:
In order to ensure that ext3 can safely resume an unlink after a crash, it actually zeros out the block pointers in the inode, whereas
ext2 just marks these blocks as unused in the block bitmaps and marks the inode as "deleted" and leaves the block pointers alone.

Your only hope is to "grep" for parts of your files that have been deleted and hope for the best.

[Source] =
http://batleth.sapienti-sat.org/projects/FAQs/ext3-faq.html

Additionally where can I get the source for Midnight Commander.

Thanks,
Uzair Lakhani

Quoted 12 lines
Signature · 10 lines
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70
370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The
Netherlands
-
To unsubscribe from this list: send the line
"unsubscribe linux-fsdevel" in
the body of a message to [email protected]
More majordomo info at 
http://vger.kernel.org/majordomo-info.html

__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com

#3 lore raw
Quoted 42 lines · UZAIR LAKHANI
Quoted 20 lines · --- Erik Mouw
Quoted 1 line · UZAIR LAKHANI

Is there any way to recover the deleted files from

a

Quoted 1 line

folder in ext3 partition. Basically in one

directory,

Quoted 1 line

I gave the command (rm -rf *) and all the files

and

Quoted 3 lines

folders of this directory are lost. How to recover these. I had once found an article regarding un-deleting ext2 files.

It *should* work the same, though the the last time I tried such a
recovery from and ext3 filesystem, ext3 made it particularly hard cause
it zeros most of the inode on delete (including ext3_inode->i_block[]).
Ext2 leaves much more information behind.

Quoted 5 lines

You could try with the undelfs feature in Midnight Commander, but first
be sure to mount the filesystem read-only in order to avoid any further
changes.

Thanks for the reply. Yes it is difficult to recover files from ext3 as compared to ext2. I found this helpful info from one website.

Q: How can I recover (undelete) deleted files from my ext3 partition?
Actually, you can't! This is what one of the developers, Andreas Dilger, said about it:
In order to ensure that ext3 can safely resume an unlink after a crash, it actually zeros out the block pointers in the inode, whereas
ext2 just marks these blocks as unused in the block bitmaps and marks the inode as "deleted" and leaves the block pointers alone.

Quoted 5 lines

Your only hope is to "grep" for parts of your files that have been deleted and hope for the best.

[Source] =
http://batleth.sapienti-sat.org/projects/FAQs/ext3-faq.html

Andreas, care to elaborate? AFAICS resetting ->i_mode to 0000, setting ->i_links_count to 0, and setting ->i_dtime should be enough to uniquely mark an inode as deleted. Doing so would make an occasional undelete much easier to recover for Joe Sixpack (and for us a lot less work to reconstruct files by scavenging the whole partition for clues).

Quoted 2 lines

Additionally where can I get the source for Midnight Commander.

IIRC it lives somewhere on ftp.gnome.org and/or on ftp.gnu.org. Look for mc-XXX.tar.gz. It started its life as a text based file manager similar to the Norton Commander. Later on it was used by the Gnome project to make a graphical file manager (gmc) which was later on replaced by Nautilus.

Erik

Signature · 2 lines
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands
#4 lore raw
Quoted 5 lines · Erik Mouw

Andreas, care to elaborate? AFAICS resetting ->i_mode to 0000, setting ->i_links_count to 0, and setting ->i_dtime should be enough to uniquely mark an inode as deleted. Doing so would make an occasional undelete much easier to recover for Joe Sixpack (and for us a lot less work to reconstruct files by scavenging the whole partition for clues).

Yes, while this would mark the _inode_ unused, it does nothing to mark the blocks unused in the block bitmaps (of which there may be many thousand/million in a single file). So ext3 needs to be able to mark these bitmap blocks unused when the file is unlinked/truncated. Otherwise each crash would leak blocks that can't be recovered except by full e2fsck.

There is another mechanism ext3 could potentially use, wherein it walks the whole inode in advance of the truncate and creates a (potentially) very large transaction handle to do the bitmap updates in a single shot (and also reducing the amount of IO needed for an unlink by 96%), but nobody has ever cared enough about it to work on implementing this.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

#5 lore raw

Hi,

Quoted 6 lines · Andreas Dilger

There is another mechanism ext3 could potentially use, wherein it walks the whole inode in advance of the truncate and creates a (potentially) very large transaction handle to do the bitmap updates in a single shot (and also reducing the amount of IO needed for an unlink by 96%), but nobody has ever cared enough about it to work on implementing this.

Trouble is, there's no guarantee that that transaction would actually fit into the journal. Most of the time it will, but if it doesn't, then we deadlock or risk data corruption.

--Stephen

#6 lore raw
Quoted 11 lines · Stephen C. Tweedie
Quoted 6 lines · Andreas Dilger

There is another mechanism ext3 could potentially use, wherein it walks the whole inode in advance of the truncate and creates a (potentially) very large transaction handle to do the bitmap updates in a single shot (and also reducing the amount of IO needed for an unlink by 96%), but nobody has ever cared enough about it to work on implementing this.

Trouble is, there's no guarantee that that transaction would actually fit into the journal. Most of the time it will, but if it doesn't, then we deadlock or risk data corruption.

Is there some way to determine in advance if a transaction fits into the journal? If so, we could unlink using Andreas's proposal if the journal allows, and fall back to the old method if not.

Erik

Signature · 2 lines
+-- Erik Mouw -- www.harddisk-recovery.com -- +31 70 370 12 90 --
| Lab address: Delftechpark 26, 2628 XH, Delft, The Netherlands
#7 lore raw

Hi,

Quoted 6 lines · Erik Mouw
Quoted 3 lines

Trouble is, there's no guarantee that that transaction would actually fit into the journal. Most of the time it will, but if it doesn't, then we deadlock or risk data corruption.

Is there some way to determine in advance if a transaction fits into the journal?

For truncate/delete, no, not easily. Or rather, it's possible, but only for trivially short files. The trouble is that we can't assume that all of the file's blocks are on the same block groups, so each block in the file is potentially an update to a new group descriptor and a new block bitmap (ie. 2 blocks of journal per block of file.)

That's hugely pessimistic, of course, but it is the genuine worst-case scenario and we have to be prepared for it. We only work out that we need less once we actually start walking the file's indirect tree, at which point the truncate is already under way.

We _could_ walk the tree twice, but that would be unnecessarily expensive, especially for large files.

--Stephen

#8 lore raw
Quoted 6 lines · Stephen C. Tweedie
Quoted 4 lines · Erik Mouw
Quoted 3 lines

Trouble is, there's no guarantee that that transaction would actually fit into the journal. Most of the time it will, but if it doesn't, then we deadlock or risk data corruption.

Quoted 8 lines
Quoted 2 lines

Is there some way to determine in advance if a transaction fits into the journal?

For truncate/delete, no, not easily. Or rather, it's possible, but only for trivially short files. The trouble is that we can't assume that all of the file's blocks are on the same block groups, so each block in the file is potentially an update to a new group descriptor and a new block bitmap (ie. 2 blocks of journal per block of file.)

Actually, the upper limit is the number of block groups in the filesystem. In many cases this would be a sufficient upper bound that could be checked without any effort. Given that the default journal size is now 128MB (32k blocks) this would guarantee full-file truncates in the worst case for up (to 32k blocks / 4 / (1 + 1/128 blocks/group)) = 8128 groups = 1016GB before we even have to consider walking the file. We could also work out the worst-case truncate by the number of blocks in the file.

Quoted 7 lines

That's hugely pessimistic, of course, but it is the genuine worst-case scenario and we have to be prepared for it. We only work out that we need less once we actually start walking the file's indirect tree, at which point the truncate is already under way.

We _could_ walk the tree twice, but that would be unnecessarily expensive, especially for large files.

That was actually my thought. I don't think it is expensive, given the current implementation already has to read all of these blocks from disk (in reverse order, I might add) and then _write_ to each of the indirect blocks (1/1024 of the whole file size) to zero them out.

Instead, we could walk the file tree in forward order, doing async readahead for the indirect, dindirect, tindirect blocks, then a second batch of readahead for all of the indirect blocks from dindirect, and dindirect blocks from tindirect, rinse repeat. In the meantime we could walk the blocks and count the block groups affected (= transaction size) while waiting for the next batch of blocks to complete IO. Since we need to read these blocks in any case, doing the readahead efficiently will likely improve performance of this step.

Rather than hurt performance, I think this will actually improve the truncate performance because we don't need to do ANY indirect block writes, removing 1 block write per 1MB of file space, only doing the (already required) write to the group descriptor and bitmap. Given that we try hard to do contiguous allocations for files this will usually be a relatively small number of blocks, as few as 1/128MB of the file size (group size limit). If we do the file walk, instead of being completely pessimistic, we can also reduce the pressure on journal flushes, which I suspect would be more costly than the walk itself.

The fact that this also fixes undelete was actually a side-effect, IMHO.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

#9 lore raw
In reply to Stephen C. Tweedie
Quoted 9 lines · Stephen C. Tweedie
Quoted 6 lines · Andreas Dilger

There is another mechanism ext3 could potentially use, wherein it walks the whole inode in advance of the truncate and creates a (potentially) very large transaction handle to do the bitmap updates in a single shot (and also reducing the amount of IO needed for an unlink by 96%), but nobody has ever cared enough about it to work on implementing this.

Quoted 3 lines

Trouble is, there's no guarantee that that transaction would actually fit into the journal. Most of the time it will, but if it doesn't, then we deadlock or risk data corruption.

The only way to do this would be to figure out how how much space is needed in advance, see if there's enough room, and if there isn't, to fall back the current strategy of doing a partial truncate. A lot of complexity to speed up unlink and make it possible to recover from unlinked files, but it's doable. No one with the skills to implement it has stepped up to date, however. :-)

- Ted