xfs_repair on Debian testing 64bits

Hi,

i have a device with 5,3To under a debian (sarge) 32bits server.

When i try a xfs_repair (2.8.20) on 5,3To i have a message :

xfs_repair: libxfs_initbuf can't memalign 4096 bytes: Cannot allocate memory [ ... ]

i tried mounting the device on a 64 bits (Debian testing) and with xfs_repair 2.8.20 there is no problem with memory but is very long (12 hours between phase 1 and phase 6)

when i use xfs_repair on 32 bits i saw 8 thread but on 64 bits just only on thread.

i try to force 8 thread with -o thread=8

and i have this message :

xfs_repair -o thread=8 /dev/sdb1
- creating 8 worker thread(s)

but with ps xa | grep repair i see only one thread.

have an idea to have several thread ?

Thanks

Patrick

IMHO, you'll need 'ps xaH'

cheers.
- Christian

Yes H for show threads as if they were processes

with ps xa under debian sarge i saw the threads

Thanks

Patrick

Out of memory. Not surprising - 5.6To of filesystem could take between 10-20GB of RAM to repair successfully. Can't do that on a 32bit machine.

Repair duration is determined by the number of inodes in the filesystem. Given the runtime you are reporting, I'd say you've got millions (perhaps even tens of milllions) of inodes in the filesystem. Is that correct?

Cheers,

Dave.

Dave Chinner
Principal Engineer
SGI Australian Software Group

patrick.noel> Hi, i have a device with 5,3To under a debian patrick.noel> (sarge) 32bits server. When i try a xfs_repair patrick.noel> (2.8.20) on 5,3To i have a message : xfs_repair:
patrick.noel> libxfs_initbuf can't memalign 4096 bytes: Cannot patrick.noel> allocate memory

http://OSS.SGI.com/archives/linux-xfs/2005-08/msg00045.html

«From some quick tests I just ran, for 32bit binaries
xfs_check needs around 1GiB RAM per TiB of filesystem plus about 100MiB RAM per 1million inodes in the filesystem (more if you have lots of fragmented files). Double this for 64bit binaries. e.g. it took 1.5GiB RAM for 32bit xfs_check and 2.7GiB RAM for a 64bit xfs_check on a 1.1TiB filesystem with 3million inodes in it.»

And from a message that is missing from the list archive:

«To successfully check or run repair on a multi-terabyte
filesystem, you need:

- a 64bit machine
- a 64bit xfs_repair/xfs_check binary
- ~2GB RAM per terabyte of filesystem
- 100-200MB of RAM per million inodes in the filesystem.

xfs_repair will usually use less memory than this, but these numbers give you a ballpark figure for what a large filesystem that is > 80% full can require to repair.

FWIW, last time this came up internally, the 29TB filesystem in question took ~75GB of RAM+swap to repair.»

patrick.noel> [ ... ] i tried mounting the device on a 64 bits patrick.noel> (Debian testing) and with xfs_repair 2.8.20 there patrick.noel> is no problem with memory but is very long (12 patrick.noel> hours between phase 1 and phase 6) [ ... ]

That seems to me quite fast, like 500GB per hour; you don't say how many files, but assuming say 50KB/file that's 10M files/hour.

For comparison sometimes it takes more than two months to 'fsck' a 1.5TB filesystem with 'ext3':

http://UKAI.org/b/log/debian/snapshot/1_month_fsck-2005-07-22-00-00.html
http://UKAI.org/b/log/debian/snapshot/fsck_completed_but-2005-09-04-15-00.html

A rarely appreciated aspect of 'fsck' is that check and repair of an error-free filesystem is usually *much* faster than for one with errors. From some small scale tests I did some time ago:

http://WWW.sabi.co.UK/blog/anno06-2nd.html#060424b

«If one extrapolates linearly from the larger filesystem, which
has an 800KiB average file size, a filesystem with 6.5TiB of data in 8.5M inodes will take at least 50 minutes to check, and one with 65TiB of data in around 85M inodes at least 11 hours. If one extrapolates from the smaller filesystem, which has a 16KiB average file size, those times must be at least 4 times larger.

Again, these are optimal times, assuming that there the filesystem is freshly restored and there are no errors. One can imagine that recovery in a filesystem with damage and fragmentation be a lot slower.»

for 5,3To

	2GB * 5,3 = 10,6Go
	
	100MB/million inodes * 45 = 4,5Go

Total = 16Go Ram

i presume the number of inodes is the number of used inodes because the number of inode on 5,3To is 5119 million.

The goal is to use a device of 15To with the same data, consider the amount total of RAM at 45Go.

I will go to the store of the corner to buy RAM :)

Thanks

Patrick