disk full or not? you decide...
hello,
i am not a subscriber to the list but i need the help of anyone on the list who might have some insite into this problem. please remember to cc me on your reply. thanks!
we are using reiserfs on a system running 2.4.3 (i think i put all of the relavant patches into the kernel before i built it then...) we have a partition that is used when encoding ripped songs and storing large files such as video and music. we noticed recently that the partition reported itself as being full. after a reboot the system reported having 6G freed. now again after a day of use the space has dissappered. df now returns:
----
Filesystem Size Used Avail Use% Mounted on /dev/hda5 706M 229M 477M 32% / /dev/hda2 55M 36M 19M 65% /boot /dev/hda7 1.0G 95M 932M 9% /usr/local /dev/hda8 26G 22G 3.7G 85% /Assets
----
however if you run du on the /Assets dir you get:
----
8.3G /Assets
----
does anyone know why this is happening? our guess is that the logs to reiser are getting quite large. how do we flush them and force a garbage collection? we save and remove several large files on this partition as the system is running. therefore, i figure that the space is kept around till the log is flushed in case it is needed for replaying the journal. am i totaly off?
i would like to upgrade the kernel but we have several third party dependencies that keep us from doing that on a fast enough pace to keep up with the 2.4.x series.
thanks again for any help that can be given. remember to cc me in responces.
Signature · 2 lines
shawn veader --oOo-- linux os developer [email protected] | http://www.zapmedia.com
Quoted 6 lines · Shawn Veader
does anyone know why this is happening? our guess is that the logs to reiser are getting quite large. how do we flush them and force a garbage collection? we save and remove several large files on this partition as the system is running. therefore, i figure that the space is kept around till the log is flushed in case it is needed for replaying the journal. am i totaly off?
No, space should be available right away, and the journal have fixed size (32 MB pr default)
Most likely the problem is caused by a big file (or more files) beeing deleted but some program still keeping it open. Then the space can not be reused until that program closes the file.
You can get a list of deleted files that are still open with:
ls -l /proc/*/fd | grep deleted
Signature · 2 lines
Ragnar Kjorstad Big Storage
Quoted 11 lines · Shawn Veader
we are using reiserfs on a system running 2.4.3 we noticed recently that the partition reported itself as being full. after a reboot the system reported having 6G freed. now again after a day of use the space has dissappered.
----
does anyone know why this is happening? our guess is that the logs to reiser are getting quite large. how do we flush them and force a garbage collection? we save and remove several large files on this partition as the system is running. therefore, i figure that the space is kept around till the log is flushed in case it is needed for replaying the journal. am i totaly off?
The problem is that something is keeping these files open, and the space cannot be freed until the process exits (or closes the files). You can use lsof to tell you which files are open, and which process is using them. If it is your own code that is causing this problem, you need to ensure that you close all of the files that you open when you are done with them.
The reiserfs journal is a fixed size, so it cannot be that.
Note also that on reiserfs, if you have such a process which keeps files open after they are deleted and then you have a crash, the file is "orphaned" and the space is "lost" until you run reiserfsck again. It may be that Chris Mason's patch for this is in the latest kernels, but it may not be, and it might not be in the kernel you are running.
Cheers, Andreas
Signature · 3 lines
Andreas Dilger \ "If a man ate a pound of pasta and a pound of antipasto,
\ would they cancel out, leaving him still hungry?"
http://www-mddsp.enel.ucalgary.ca/People/adilger/ -- DogbertQuoted 5 lines · Andreas Dilger
Note also that on reiserfs, if you have such a process which keeps files open after they are deleted and then you have a crash, the file is "orphaned" and the space is "lost" until you run reiserfsck again. It may be that Chris Mason's patch for this is in the latest kernels, but it may not be, and it might not be in the kernel you are running.
Chris, Hans, is this problem still in the reiserfs in the current kernel or has it already been fixed ?
If it's still there, is a patch available which can be considered stable ?
Rik
--
Virtual memory is like a game you can't win; However, without VM there's truly nothing to lose...
http://www.surriel.com/ http://distro.conectiva.com/
Send all your spam to [email protected] (spam digging piggy)
Quoted 10 lines · On Wednesday, July 11, 2001 07:10:27 PM -0300 Rik van Riel…
Quoted 5 lines · Andreas Dilger
Note also that on reiserfs, if you have such a process which keeps files open after they are deleted and then you have a crash, the file is "orphaned" and the space is "lost" until you run reiserfsck again. It may be that Chris Mason's patch for this is in the latest kernels, but it may not be, and it might not be in the kernel you are running.
Chris, Hans, is this problem still in the reiserfs in the current kernel or has it already been fixed ?
Vladimir Saveliev has a patch that is doing well in internal testing. I had wanted to intergrate it some nested transaction stuff, but it didn't have the same advantages as it did under 2.2.x.
They will probably send it out for wider testing very soon.
Just to clarify, you need to have a crash or unclean un -chris
Quoted 30 lines · Shawn Veader
hello,
i am not a subscriber to the list but i need the help of anyone on the list who might have some insite into this problem. please remember to cc me on your reply. thanks!we are using reiserfs on a system running 2.4.3 (i think i put all of the relavant patches into the kernel before i built it then...) we have a partition that is used when encoding ripped songs and storing large files such as video and music. we noticed recently that the partition reported itself as being full. after a reboot the system reported having 6G freed. now again after a day of use the space has dissappered. df now returns:
----Filesystem Size Used Avail Use% Mounted on /dev/hda5 706M 229M 477M 32% / /dev/hda2 55M 36M 19M 65% /boot /dev/hda7 1.0G 95M 932M 9% /usr/local /dev/hda8 26G 22G 3.7G 85% /Assets----
however if you run du on the /Assets dir you get:
----
8.3G /Assets
----
does anyone know why this is happening? our guess is that the logs to reiser are getting quite large. how do we flush them and force a garbage collection? we save and remove several large files on this partition as the system is running. therefore, i figure that the space is kept around till the log is flushed in case it is needed for replaying the journal. am i totaly off?
Quoted 15 lines
i would like to upgrade the kernel but we have several third party dependencies that keep us from doing that on a fast enough pace to keep up with the 2.4.x series.
thanks again for any help that can be given. remember to cc me in responces.
Signature · 7 lines
shawn veader --oOo-- linux os developer [email protected] | http://www.zapmedia.com - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Try this to see if it is a large file left on the partition: du -a | sort
I've had a problem similar to that where something spazzed out and wrote about 10 gigs to a log file. Still haven't figured out what caused that.
Signature · 3 lines
-Aaron Don't hate yourself in the morning, sleep till noon
Quoted 5 lines · Aaron Smith
Quoted 5 lines
however if you run du on the /Assets dir you get:
----
8.3G /Assets
----
does anyone know why this is happening? our guess is that the logs
Check out the FAQ on www.namesys.com, #17.
Quoted 19 lines · Ragnar KjЬrstad
Quoted 6 lines · Shawn Veader
does anyone know why this is happening? our guess is that the logs to reiser are getting quite large. how do we flush them and force a garbage collection? we save and remove several large files on this partition as the system is running. therefore, i figure that the space is kept around till the log is flushed in case it is needed for replaying the journal. am i totaly off?
No, space should be available right away, and the journal have fixed size (32 MB pr default)
Most likely the problem is caused by a big file (or more files) beeing deleted but some program still keeping it open. Then the space can not be reused until that program closes the file.
You can get a list of deleted files that are still open with:
ls -l/proc/*/fd| grep deleted
Quoted 8 lines
--
Ragnar Kjorstad
Big Storage
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
this is probably due to a fixed bug addressed in a not yet released bugfix.
We are about to send it into the ac series (probably Sunday). If you run fsck it should free up the space, but run a backup first.
Hans
Re: disk full or not? Disk space dissapear :-(
In reply to Shawn VeaderQuoted 5 lines · Shawn Veader
Date: Wed, 11 Jul 2001 15:30:11 -0400
From: Shawn Veader <[email protected]>
To: [email protected]
Subject: disk full or not? you decide...
Quoted 8 lines
we are using reiserfs on a system running 2.4.3 (i think i put all of the relavant patches into the kernel before i built it then...) we have a partition that is used when encoding ripped songs and storing large files such as video and music. we noticed recently that the partition reported itself as being full. after a reboot the system reported having 6G freed. now again after a day of use the space has dissappered. df now returns:
----
I have had det samme problem with 2.4.1, 2.4.3, 2.4.4 but I do not use reiserfs, just normal ext2.
It does not happen all the time but sometimes when I delete a file bigger than 2GB, the space used by the file is not freed, This space dissapears from the system and the only way to get it back is to reboot the machine.
No errors, no problems with the rest of the system, the space just dissapear.
We use to work with files up to 7-9GB and the only thing I can notice when this happens is that the time it takes to erase the file is very, very short (under a second) when the space dissapear and it takes a few seconds to erase a file of this size when everything works OK.
Any ideas?
Sincerely
Rafael Martinez.