XFS defragmentation issue
Hi, using ubuntu 9.04, kernel version 2.6.28-18 and xfsprog v2.10.2-1
I'm trying to defragment a drive. If I use xfs_db to check for fragmentation it shows for example that a file has 20k+ fragments:
inode 578505506 actual 23240 ideal 1
mapping the inode to the file and running xfs_frs states that the file is fully defragmented.
Also running xfs_frs says the drive is defragmented while xfs_db for example reports 300k fragments.
Which is right? and is it possible to defragment the file assuming it is fragmented so badly, if so how?
Thanks
John
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
John Lister put forth on 9/10/2010 9:41 AM:
Quoted 5 lines
Hi, using ubuntu 9.04, kernel version 2.6.28-18 and xfsprog v2.10.2-1
I'm trying to defragment a drive. If I usexfs_dbto check for fragmentation it shows for example that a file has 20k+ fragments:
inode 578505506 actual 23240 ideal 1
Quoted 8 lines
mapping the inode to the file and running
xfs_frsstates that the file is fully defragmented.Also running
xfs_frssays the drive is defragmented whilexfs_dbfor example reports 300k fragments.Which is right? and is it possible to defragment the file assuming it is fragmented so badly, if so how?
Try unmounting and remounting the filesystem, and see if the various tools all report the same thing afterwards. This solved the exact same problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs 2.9.8.
Signature · 6 lines
Stan _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Stan Hoeppner wrote on 9/10/2010 14:00
Quoted 6 lines
Quoted 6 lines · John Lister
Hi, using ubuntu 9.04, kernel version 2.6.28-18 and xfsprog v2.10.2-1 I'm trying to defragment a drive. If I use
xfs_dbto check for fragmentation it shows for >>example that a file has 20k+ fragments:
inode 578505506 actual 23240 ideal 1mapping the inode to the file and running
xfs_frsstates that the file is fully defragmented.
Quoted 8 lines
Quoted 3 lines
Also running
xfs_frssays the drive is defragmented whilexfs_dbfor example reports 300k fragments.Which is right? and is it possible to defragment the file assuming it is fragmented so badly, if so how?
Try unmounting and remounting the filesystem, and see if the various tools all report the same thing afterwards. This solved the exact same problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs 2.9.8.
Cheers, that got rid of most of it, there is still a slight discrepency (50 extra fragments) which I can live with.
John
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 9 lines · John Lister
Stan Hoeppner wrote on 9/10/2010 14:00
Quoted 5 lines
Quoted 1 line · John Lister
Try unmounting and remounting the filesystem, and see if the various tools all report the same thing afterwards. This solved the exact same problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs 2.9.8.
Cheers, that got rid of most of it, there is still a slight discrepency (50 extra fragments) which I can live with.
xfs_db used buffered IO on the block device, which is not coherent with the filesystem. If you are using it on an active filesystem, then running "echo 1 > /proc/sys/vm/drop_caches" before you run xfs_db should make it read from disk at least once....
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 7 lines · Dave Chinner
Quoted 6 lines · John Lister
Stan Hoeppner wrote on 9/10/2010 14:00
Quoted 5 lines
Quoted 1 line · John Lister
Try unmounting and remounting the filesystem, and see if the various tools all report the same thing afterwards. This solved the exact same problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs 2.9.8.
Quoted 6 lines
Quoted 2 lines
Cheers, that got rid of most of it, there is still a slight discrepency (50 extra fragments) which I can live with.
xfs_dbused buffered IO on the block device, which is not coherent with the filesystem. If you are using it on an active filesystem, then running "echo 1>/proc/sys/vm/drop_caches" before you runxfs_dbshould make it read from disk at least once....
Ah, that makes more sense, I'll give it a go.
Thanks
John
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Dave Chinner put forth on 9/11/2010 3:23 AM:
Quoted 8 lines · John Lister
Quoted 7 lines
Stan Hoeppner wrote on 9/10/2010 14:00
Quoted 5 lines
Quoted 1 line · John Lister
Try unmounting and remounting the filesystem, and see if the various tools all report the same thing afterwards. This solved the exact same problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs 2.9.8.
Quoted 7 lines
Quoted 2 lines
Cheers, that got rid of most of it, there is still a slight discrepency (50 extra fragments) which I can live with.
xfs_dbused buffered IO on the block device, which is not coherent with the filesystem. If you are using it on an active filesystem, then running "echo 1 >/proc/sys/vm/drop_caches" before you runxfs_dbshould make it read from disk at least once....
That's good to know Dave. Thanks. I created a short-name script a while back with "echo 3 > /proc/sys/vm/drop_caches" merely so I could "clear the baffles" now and then. Looks like it now has multiple uses (if I can just remember to use it in this context).
Out of curiosity, who here schedules automatic xfs_fsr runs, and with what frequency? Currently I just run it manually now and then when performance starts to seem sluggish.
Thanks.
Signature · 6 lines
Stan _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 16 lines · Stan Hoeppner
Dave Chinner put forth on 9/11/2010 3:23 AM:
Quoted 15 lines · John Lister
Quoted 9 lines
Stan Hoeppner wrote on 9/10/2010 14:00
Quoted 5 lines
Quoted 1 line · John Lister
Try unmounting and remounting the filesystem, and see if the various tools all report the same thing afterwards. This solved the exact same problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs 2.9.8.
Cheers, that got rid of most of it, there is still a slight discrepency (50 extra fragments) which I can live with.
xfs_dbused buffered IO on the block device, which is not coherent with the filesystem. If you are using it on an active filesystem, then running "echo 1 >/proc/sys/vm/drop_caches" before you runxfs_dbshould make it read from disk at least once....
I wonder if xfs_db shouldn't use by default direct I/O, or at least take a flag to allow it to do direct I/O only against the blocks it needs. Dropping the entire caches on a big box is not nice :)
Quoted 8 lines
That's good to know Dave. Thanks. I created a short-name script a while back with "echo 3 >
/proc/sys/vm/drop_caches" merely so I could "clear the baffles" now and then. Looks like it now has multiple uses (if I can just remember to use it in this context).Out of curiosity, who here schedules automatic
xfs_fsrruns, and with what frequency? Currently I just run it manually now and then when performance starts to seem sluggish.
This is my setup, on a machine with multiple XFS filesystems:
$ cat /etc/cron.d/xfsfsr PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin
0 01 * * 7 root xfs_fsr -v
I've set it up a few years back and, except for a few rare bugs in 2.6.x which are then fixed in 2.6.x.y, I've had no issues.
regards,
iustin
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 13 lines · Iustin Pop
Quoted 12 lines · Stan Hoeppner
Dave Chinner put forth on 9/11/2010 3:23 AM:
Quoted 11 lines · John Lister
Quoted 9 lines
Stan Hoeppner wrote on 9/10/2010 14:00
Quoted 5 lines
Quoted 1 line · John Lister
Try unmounting and remounting the filesystem, and see if the various tools all report the same thing afterwards. This solved the exact same problem for me very recently, though I'm on kernel 2.6.34.1 and xfsprogs 2.9.8.
Cheers, that got rid of most of it, there is still a slight discrepency (50 extra fragments) which I can live with.
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
xfs_dbused buffered IO on the block device, which is not coherent with the filesystem. If you are using it on an active filesystem, then running "echo 1 >/proc/sys/vm/drop_caches" before you runxfs_dbshould make it read from disk at least once....I wonder if
xfs_dbshouldn't use by default direct I/O, or at least take a flag to allow it to do direct I/O only against the blocks it needs. Dropping the entire caches on a big box is not nice :)
True - I need to look at whether posix_fadvise(POSIX_FADV_DONTNEED) will clear the bdev pages, and if so that is the easiest solution. Rewriting xfs_db to use direct IO is a pretty major undertaking...
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Le Sat, 11 Sep 2010 14:38:48 -0500 vous écriviez:
Quoted 3 lines
That's good to know Dave. Thanks. I created a short-name script a while back with "echo 3 >
/proc/sys/vm/drop_caches" merely so I could "clear the baffles" now and then.
But isn't it somewhat risky? isn't there any risk of data loss by dropping cache at any moment?
Signature · 11 lines
------------------------------------------------------------------------
Emmanuel Florac | Direction technique
| Intellique
| <[email protected]>
| +33 1 78 94 84 02
------------------------------------------------------------------------
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfsEmmanuel Florac put forth on 9/12/2010 3:43 AM:
Quoted 8 lines
Le Sat, 11 Sep 2010 14:38:48 -0500 vous écriviez:
Quoted 3 lines
That's good to know Dave. Thanks. I created a short-name script a while back with "echo 3 >
/proc/sys/vm/drop_caches" merely so I could "clear the baffles" now and then.But isn't it somewhat risky? isn't there any risk of data loss by dropping cache at any moment?
From: http://linux-mm.org/Drop_Caches
This is a non-destructive operation and will only free things that are completely unused. Dirty objects will continue to be in use until written out to disk and are not freeable. If you run "sync" first to flush them out to disk, these drop operations will tend to free more memory.
Signature · 6 lines
Stan _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 3 lines · Stan Hoeppner
Out of curiosity, who here schedules automatic
xfs_fsrruns, and with what frequency? Currently I just run it manually now and then when performance starts to seem sluggish.
I run it daily during the week, at a time when load is practically nil, on two boxes with 24TB storage each, divided into six parts of about 4GB. A 10-pass xfs_fsr run usually completes in under one hour, except on monday, when it takes three to four hours.
I don't have any stats on the net effect, though.
Regards,
roel
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs