xfs_ncheck (actually xfs_db) eats a lot of memory and is killed
Hi,
A few weeks ago I described problem with my file system (caused by a bug in kernel 2.6.17) - http://oss.sgi.com/archives/xfs/2006-11/msg00271.html
Recently I've got rescue CD with xfs_progs in version 2.8.11 and tried to repair it. xfs_repair gave me about 150 files with names like inode numbers in /lost+found and 1500 "normal" named files in its subdirectory.
I've tried to use "xfs_ncheck -i inode /dev/hdX" to got names corresponding with specified inode(s), but program had been runing several minutes, xfs_db had eaten all available memory (768MB) and was killed by system (whole system hung too). The second time I killed it (kill -15) when it ate whole available memory. My file system has about 6GB and is filled with 95%.
Am I doing something wrong with xfs_db?
Is there any easier way to restore my files?
Btw, xfs_check shows two lines:
link count mismatch for inode 1572882 (name ?), nlink 16, counted 15 link count mismatch for inode 2102297 (name ?), nlink 2, counted 3 Can it be reason for strange xfs_db behavior?
Thanks for help
Marcin
Quoted 18 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Marcin Zajaczkowski
Sent: Sunday, 17 December 2006 4:58 AM
To: [email protected]
Subject:xfs_ncheck(actuallyxfs_db) eats a lot of memory and is killedHi,
A few weeks ago I described problem with my file system (caused by a bug
in kernel 2.6.17) -
http://oss.sgi.com/archives/xfs/2006-11/msg00271.html
Quoted 13 lines
Recently I've got rescue CD with
xfs_progsin version 2.8.11 and tried
to repair it.xfs_repairgave me about 150 files with names like inode
numbers in /lost+found and 1500 "normal" named files in its subdirectory.
I've tried to use "xfs_ncheck-i inode/dev/hdX" to got names corresponding with specified inode(s), but program had been runing several minutes,xfs_dbhad eaten all available memory (768MB) and was
killed by system (whole system hung too). The second time I killed it (kill -15) when it ate whole available memory.
My file system has about 6GB and is filled with 95%.
This is the second report of a smallish filesystem using all of the system's memory (the other being xfs_repair). Hopefully when I diagnose the problem with the previous report, I can fix the same issue with xfs_db and your filesystem.
If it at all possible, can you xfs_copy the offending filesystem to a file and compress it and make it available to me to find/fix the problem.
Quoted 2 lines
Am I doing something wrong with
xfs_db?
Is there any easier way to restore my files?
Your files have been restored as much as possible. You'll have to work out what is in lost+found and move them back to their appropriate locations.
Quoted 4 lines
Btw,
xfs_checkshows two lines:
link count mismatch for inode 1572882 (name ?), nlink 16, counted 15 link count mismatch for inode 2102297 (name ?), nlink 2, counted 3 Can it be reason for strangexfs_dbbehavior?
No, this shouldn't be causing the xfs_db strange behaviour, but the nlink count mismatch is a problem with xfs_repair 2.8.11. Try and use xfs_repair 2.8.16 and later to fix this issue. Older versions before 2.8.11 will also fix the nlink issue.
Quoted 1 line · Barry Naujok
Quoted 1 line
Subject:
xfs_ncheck(actuallyxfs_db) eats a lot of memory and is killed
(...)
Quoted 4 lines
Quoted 3 lines
Recently I've got rescue CD with
xfs_progsin version 2.8.11 and tried to repair it.xfs_repairgave me about 150 files with names like inode numbers in /lost+found and 1500 "normal" named files in its subdirectory.
I've tried to use "xfs_ncheck-i inode/dev/hdX" to got names corresponding with specified inode(s), but program had been runing several minutes,xfs_dbhad eaten all available memory (768MB) and was killed by system (whole system hung too). The second time I killed it (kill -15) when it ate whole available memory. My file system has about 6GB and is filled with 95%.
Quoted 8 lines
This is the second report of a smallish filesystem using all of the system's memory (the other being
xfs_repair). Hopefully when I diagnose the problem with the previous report, I can fix the same issue withxfs_dband your filesystem.If it at all possible, can you
xfs_copythe offending filesystem to a file and compress it and make it available to me to find/fix the problem.
Hmm, it won't be so easy. Compressed dump of a fielsystem before repair (I had to use dd, because xfsdump refused to cooperate) has 2,5GB. Damaged files were (probably) only in one directory /usr/bin. Maybe I could reduce size of the image excluding few other directories (in xfsdump)? Do you think that error would still occur?
Quoted 6 lines
Quoted 2 lines
Am I doing something wrong with
xfs_db?
Is there any easier way to restore my files?Your files have been restored as much as possible. You'll have to work out what is in lost+found and move them back to their appropriate locations.
With files with "normal" names there is no problem. There are all from one directory (/usr/bin), but I'm unable (without xfs_ncheck) to map files with inode-like names with their normal names (150+ files). I tried with xfs_db and the way described in your FAQ, but I had some problems too.
Quoted 9 lines
Quoted 4 lines
Btw,
xfs_checkshows two lines:
link count mismatch for inode 1572882 (name ?), nlink 16, counted 15 link count mismatch for inode 2102297 (name ?), nlink 2, counted 3 Can it be reason for strangexfs_dbbehavior?No, this shouldn't be causing the
xfs_dbstrange behaviour, but the nlink count mismatch is a problem withxfs_repair2.8.11. Try and usexfs_repair2.8.16 and later to fix this issue. Older versions before 2.8.11 will also fix the nlink issue.
I have some old system rescue CD with xfs_progs from line 2.7.x. Would it be ok?
Regards
Marcin
Quoted 19 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Marcin Zajaczkowski
Sent: Monday, 18 December 2006 7:15 PM
To: [email protected]
Subject: Re:xfs_ncheck(actuallyxfs_db) eats a lot of memory and is killedQuoted 2 lines · Barry Naujok
Quoted 2 lines
Btw,
xfs_checkshows two lines:
link count mismatch for inode 1572882 (name ?), nlink 16,counted 15
Quoted 5 lines
Quoted 2 lines
link count mismatch for inode 2102297 (name ?), nlink 2, counted 3 Can it be reason for strange
xfs_dbbehavior?No, this shouldn't be causing the
xfs_dbstrange behaviour, but the nlink count mismatch is a problem withxfs_repair2.8.11.
Quoted 7 lines
Try and use
Quoted 2 lines
xfs_repair2.8.16 and later to fix this issue. Older versions before 2.8.11 will also fix the nlink issue.I have some old system rescue CD with
xfs_progsfrom line 2.7.x. Would
it be ok?
Yes, that should be fine AFAIK.
Barry.
Quoted 12 lines · Marcin Zajączkowski
Quoted 1 line · Barry Naujok
Quoted 1 line
Subject:
xfs_ncheck(actuallyxfs_db) eats a lot of memory and is killed(...)
Quoted 9 lines
Quoted 3 lines
Recently I've got rescue CD with
xfs_progsin version 2.8.11 and tried to repair it.xfs_repairgave me about 150 files with names like inode numbers in /lost+found and 1500 "normal" named files in its subdirectory.
I've tried to use "xfs_ncheck-i inode/dev/hdX" to got names corresponding with specified inode(s), but program had been runing several minutes,xfs_dbhad eaten all available memory (768MB) and was killed by system (whole system hung too). The second time I killed it (kill -15) when it ate whole available memory. My file system has about 6GB and is filled with 95%.This is the second report of a smallish filesystem using all of the system's memory (the other being
xfs_repair). Hopefully when I diagnose the problem with the previous report, I can fix the same issue withxfs_dband your filesystem.
Quoted 7 lines
Quoted 3 lines
If it at all possible, can you
xfs_copythe offending filesystem to a file and compress it and make it available to me to find/fix the problem.Hmm, it won't be so easy. Compressed dump of a fielsystem before repair (I had to use dd, because xfsdump refused to cooperate) has 2,5GB. Damaged files were (probably) only in one directory
/usr/bin. Maybe I could reduce size of the image excluding few other directories (in xfsdump)? Do you think that error would still occur?
What do you think about that?
Btw, is it possible to mount XFS filesystem from the file created by xfs_copy?
If not, is it possible to restore file system "backuped" to file (by xfs_copy)? I read in the manual that the first parameter xfs_copy has to be device (not file). xfs_restore will work with that "dump"?
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Am I doing something wrong with
xfs_db?
Is there any easier way to restore my files?Your files have been restored as much as possible. You'll have to work out what is in lost+found and move them back to their appropriate locations.
With files with "normal" names there is no problem. There are all from one directory (
/usr/bin), but I'm unable (withoutxfs_ncheck) to map files with inode-like names with their normal names (150+ files). I tried withxfs_dband the way described in your FAQ, but I had some problems too.
Currently I have to use LiveCD (/usr/bin is a quite important directory ;) ). Do you suggest to try to repair it manually (copy files with names, try to boot and try to use RPM to repair broken packages (restore missing files) or there could be an easiest way (at the xfs layer to match numbers with proper names)?
Regards
Marcin
Quoted 42 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Marcin Zajaczkowski
Sent: Tuesday, 19 December 2006 6:39 PM
To: [email protected]
Subject: Re:xfs_ncheck(actuallyxfs_db) eats a lot of memory and is killedQuoted 2 lines · Marcin Zajączkowski
Quoted 1 line · Barry Naujok
Quoted 1 line
Subject:
xfs_ncheck(actuallyxfs_db) eats a lot ofmemory and is killed
Quoted 4 lines
(...)
Quoted 3 lines
Quoted 3 lines
Recently I've got rescue CD with
xfs_progsin version 2.8.11 and tried to repair it.xfs_repairgave me about 150 files with names like inode numbers in /lost+found and 1500 "normal" namedfiles in
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
its subdirectory.
I've tried to use "xfs_ncheck-i inode/dev/hdX" to got names corresponding with specified inode(s), but program hadbeen runing
Quoted 1 line
Quoted 1 line
Quoted 1 line
several minutes,
xfs_dbhad eaten all available memory(768MB) and
Quoted 6 lines
Quoted 6 lines
Quoted 3 lines
was killed by system (whole system hung too). The second time I killed it (kill -15) when it ate whole available memory. My file system has about 6GB and is filled with 95%.
This is the second report of a smallish filesystem using all of the system's memory (the other being
xfs_repair). Hopefullywhen I diagnose
Quoted 4 lines
Quoted 4 lines
the problem with the previous report, I can fix the same issue with
xfs_dband your filesystem.If it at all possible, can you
xfs_copythe offendingfilesystem to a
Quoted 4 lines
Quoted 2 lines
file and compress it and make it available to me to find/fix the problem.
Hmm, it won't be so easy. Compressed dump of a fielsystem
before repair
Quoted 8 lines
Quoted 2 lines
(I had to use dd, because xfsdump refused to cooperate) has 2,5GB. Damaged files were (probably) only in one directory
/usr/bin. Maybe IQuoted 3 lines
could reduce size of the image excluding few other directories (in xfsdump)?
Do you think that error would still occur?What do you think about that?
xfs_dump won't recreate the filesystem that causes xfs_db to gobble up memory unfortunately, xfs_copy copies the filesystem as it is, block for block.
Quoted 6 lines
Btw, is it possible to mount XFS filesystem from the file created by
xfs_copy?
If not, is it possible to restore file system "backuped" to file (byxfs_copy)? I read in the manual that the first parameterxfs_copyhas to
be device (not file).xfs_restorewill work with that "dump"?
Yes, you can mount an XFS filesystem file using the "-o loop" mount option and specifying the file for the device.
Yes, you can xfs_copy from a file back to a device. From the man page:
"The first (source) argument must be the pathname of the device or file containing the XFS filesystem.".
xfs_dump/xfs_restore is more like tar copying all metadata, but not disk layout.
Quoted 16 lines
Quoted 4 lines
Quoted 4 lines
Quoted 2 lines
Am I doing something wrong with
xfs_db?
Is there any easier way to restore my files?Your files have been restored as much as possible. You'll
have to work
Quoted 4 lines
Quoted 2 lines
out what is in lost+found and move them back to their appropriate locations.
With files with "normal" names there is no problem. There
are all from
Quoted 1 line
one directory (
/usr/bin), but I'm unable (without
xfs_ncheck) to mapQuoted 3 lines
files with inode-like names with their normal names (150+ files). I tried with
xfs_dband the way described in your FAQ, but I had some problems too.
Quoted 6 lines
Currently I have to use LiveCD (
/usr/binis a quite important directory
;) ). Do you suggest to try to repair it manually (copy files with names, try to boot and try to use RPM to repair broken packages (restore missing files) or there could be an easiest way (at the xfs layer to match numbers with proper names)?
If the /usr/bin directory was trashed, I think you can only restore from the original packages, I don't think you'll have much luck doing it manually. (xfs_repair 2.8.10 onwards will restore the same files that the manual approach can retrieve.)
Barry Naujok wrote:
(...)
Quoted 9 lines
Quoted 2 lines
Btw, is it possible to mount XFS filesystem from the file created by
xfs_copy?
If not, is it possible to restore file system "backuped" to file (byxfs_copy)? I read in the manual that the first parameterxfs_copyhas to be device (not file).xfs_restorewill work with that "dump"?Yes, you can mount an XFS filesystem file using the "-o loop" mount option and specifying the file for the device.
Yes, you can
xfs_copyfrom a file back to a device. From the man page:
"The first (source) argument must be the pathname of the device or file containing the XFS filesystem.".
In fact. I don't know why I missed it earlier.
Thanks
Marcin