Strange case of deleting lots of files and "df" not recognizing it....(unless I umount/mount) the disk...
This isn't that big of a deal, as I'm not close to the limit, but I pruned a backup disk that was at 87% full (1.5T free), down by 3.8T. Indeed, doing a 'du' of the disk shows space is ~6-7T (/11T) but df still shows 9.5T usage.
I've done sync a few times, but the files were deleted almost a half an hour ago.
So when does it actually synchronize w/o me forcing it? (I.e. umount/mount)?
Have seen this before, and that's what brought it into 'sync'....
none of the xfs_centitimes are greater than 3000 or 30 seconds. Not a big problem that I know of, hardly a problem at all, cept that it is weird.
Curious?
-linda
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Le Sun, 24 Jun 2012 22:58:12 -0700 vous écriviez:
Quoted 3 lines
So when does it actually synchronize w/o me forcing it? (I.e. umount/mount)?
Yes, it happens sometimes and a umount/mount is needed to fix it.
Signature · 11 lines
------------------------------------------------------------------------
Emmanuel Florac | Direction technique
| Intellique
| <[email protected]>
| +33 1 78 94 84 02
------------------------------------------------------------------------
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfsQuoted 7 lines · Emmanuel Florac
Le Sun, 24 Jun 2012 22:58:12 -0700 vous écriviez:
Quoted 3 lines
So when does it actually synchronize w/o me forcing it? (I.e. umount/mount)?
Yes, it happens sometimes and a umount/mount is needed to fix it.
You may find that this will get the job done:
echo 3 > /proc/sys/vm/drop_caches
-Ben
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 6 lines · Ben Myers
Quoted 5 lines · Emmanuel Florac
Le Sun, 24 Jun 2012 22:58:12 -0700 vous écriviez:
Quoted 3 lines
So when does it actually synchronize w/o me forcing it? (I.e. umount/mount)?
Quoted 7 lines
Quoted 1 line
Yes, it happens sometimes and a umount/mount is needed to fix it.
You may find that this will get the job done:
echo 3 >/proc/sys/vm/drop_caches-Ben
Yup...it went away for a LONG time...maybe 30 seconds...
Does that mean there was that much unsynchronized data being held in memory that wasn't being written out??
It DID fix the space allocation issue.
When I read the value before changing it, it said '0'.
when it finished, I tried to set it back to '0', but got an invalid argument.??? Was it really '2' or something else?
BTW -- it had been in the weird state for 13 hours before I issued the order to drop_caches... seems like a long time not to cache data...?
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Hey Linda,
Quoted 12 lines · Linda Walsh
Quoted 11 lines · Ben Myers
Quoted 6 lines · Emmanuel Florac
Le Sun, 24 Jun 2012 22:58:12 -0700 vous écriviez:
Quoted 3 lines
So when does it actually synchronize w/o me forcing it? (I.e. umount/mount)?
Yes, it happens sometimes and a umount/mount is needed to fix it.
You may find that this will get the job done:
echo 3 >/proc/sys/vm/drop_caches
Quoted 8 lines
Quoted 1 line
-Ben
Yup...it went away for a LONG time...maybe 30 seconds...
Does that mean there was that much unsynchronized data being held in memory that wasn't being written out??
It DID fix the space allocation issue.
XFS doesn't clean up the blocks for a deleted file until the last reference on the inode goes away. Dropping caches just forces the issue.
Quoted 4 lines
When I read the value before changing it, it said '0'.
when it finished, I tried to set it back to '0', but got an invalid argument.??? Was it really '2' or something else?
Don't worry about the current value of /proc/sys/vm/drop_caches... It just retains the last value written to it but doesn't do anything with it. The only time it takes an action is when you write to it.
Quoted 2 lines
BTW -- it had been in the weird state for 13 hours before I issued the order to
drop_caches... seems like a long time not to cache data...?
Yeah. 13 hours is a long time, esp for something you just removed. ;)
Fortunately it's not the data that is cached, it's probably just a pesky dentry hanging around on the lru with a single reference on the inode keeping it active, preventing xfs from freeing up the blocks.
I've had some luck on v3.0 with the following patch... it's not yet tested on 3.5 so YMMV. Are you running an NFS server? Something that often creates anonymous dentries?
Regards,
Ben
--- fs/xfs/xfs_vnodeops.c | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+)
fs/xfs/xfs_vnodeops.c +38 -0
@@ -1203,6 +1203,41 @@ xfs_lock_two_inodes( } } +/* +* Prune any disconnected dentries from the inode. This will help to ensure +* timely teardown of the inode by unhashing all disconnected (anonymous) +* dentries that may have been added by an interface that uses filehandles like +* NFS. +* +* Here we know that there must currently be a dentry with a name on this inode +* because we're in an unlink/rmdir path, so we do not run the risk of inode +* reclaim here because we only unhash disconnected dentries. +*/ +static void +xfs_d_prune_disconnected ( +struct xfs_inode *ip) +{ +struct inode *inode =VFS_I (ip); +struct dentry *alias; + +restart: +spin_lock (&inode->i_lock); +list_for_each_entry (alias, &inode->i_dentry, d_alias) { +spin_lock (&alias->d_lock); +if (alias->d_flags &DCACHE_DISCONNECTED && + !d_unhashed (alias)) { +dget_dlock (alias); +__d_drop (alias); +spin_unlock (&alias->d_lock); +spin_unlock (&inode->i_lock); +dput (alias); + goto restart; + } +spin_unlock (&alias->d_lock); + } +spin_unlock (&inode->i_lock); +} + int xfs_remove( xfs_inode_t *dp,
@@ -1347,6 +1382,9 @@ xfs_remove(if (error) goto std_return; +if (link_zero) +xfs_d_prune_disconnected (ip); +/* * If we are using filestreams, kill the stream association. * If the file is still open it may get a new one but that
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 10 lines · Ben Myers
Yeah. 13 hours is a long time, esp for something you just removed. ;)
Fortunately it's not the data that is cached, it's probably just a pesky dentry hanging around on the lru with a single reference on the inode keeping it active, preventing xfs from freeing up the blocks.
I've had some luck on v3.0 with the following patch... it's not yet tested on 3.5 so YMMV. Are you running an NFS server? Something that often creates anonymous dentries?
----
nfs server is running but no clients (usage=rare) samba server is running and has clients (usage=24/7)...
I wasn't even able to compile the last kernel I tried...my release has been a bit screwey...
finally got perl to compile, that was another thorn in my side...
(when it rains it pours....)
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 9 lines · Linda Walsh
This isn't that big of a deal, as I'm not close to the limit, but I pruned a backup disk that was at 87% full (1.5T free), down by 3.8T. Indeed, doing a 'du' of the disk shows space is ~6-7T (/11T) but df still shows 9.5T usage.
I've done sync a few times, but the files were deleted almost a half an hour ago.
So when does it actually synchronize w/o me forcing it? (I.e. umount/mount)?
As ben said, the last reference has to go away before the space is freed. lsof is your friend.
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Le Tue, 26 Jun 2012 12:33:02 +1000 vous écriviez:
Quoted 2 lines
As ben said, the last reference has to go away before the space is freed. lsof is your friend.
I just had the same problem recently, but lsof reported zero open files on the volume. umount/mount worked though, I didn't thought the cache trick would be enough.
Signature · 11 lines
------------------------------------------------------------------------
Emmanuel Florac | Direction technique
| Intellique
| <[email protected]>
| +33 1 78 94 84 02
------------------------------------------------------------------------
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfsQuoted 8 lines · Emmanuel Florac
Le Tue, 26 Jun 2012 12:33:02 +1000 vous écriviez:
Quoted 2 lines
As ben said, the last reference has to go away before the space is freed. lsof is your friend.
I just had the same problem recently, but lsof reported zero open files on the volume. umount/mount worked though, I didn't thought the cache trick would be enough.
I think that we need a reproducable test case for this issue before we'll be able to make any progress on it, then....
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Well, in my case, may be a while, it is usually every few months when I clear out about 30% of a large (~11T) volume on my backup disk...
It only grows slowly as I make backups on it...then I delete maybe ... 60-100 files worth of backups (xfsdumps)....but for me dropping the cache did work so it may not be the same problem.
Quoted 9 lines · Dave Chinner
Quoted 7 lines · Emmanuel Florac
Le Tue, 26 Jun 2012 12:33:02 +1000 vous écriviez:
Quoted 2 lines
As ben said, the last reference has to go away before the space is freed. lsof is your friend.
I just had the same problem recently, but lsof reported zero open files on the volume. umount/mount worked though, I didn't thought the cache trick would be enough.
Quoted 6 lines
I think that we need a reproducable test case for this issue before we'll be able to make any progress on it, then....
Cheers,
Dave.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs