Optimal XFS formatting options?
Quoted 10 lines · MikeJeezy
Hi, I have a 4.9 TB iSCSI LUN on a RAID 6 array with twelve 2 TB SATA disks (4.9T is only one of the logical volumes). It will contain several million files of various sizes, but 80% of them will be less than 50 MB. I'm a novice at best and I usually just use the default #mkfs.xfs
/dev/sdx1This is server will be write heavy for about 8 hours a night, but every morning there are many reads to the disk. There is rarely a time where it will be write heavy and read heavy at the same time. Are there other XFS format options that I could use to optimize performance?
sunit=value
This is used to specify the stripe unit for a RAID device or a logical volume. The value has to be specified in 512-byte block units. Use the su suboption to specify the stripe unit size in bytes. This suboption ensures that data allocations will be stripe unit aligned when the current end of file is being extended and the file size is larger than 512KiB. Also inode allocations and the internal log will be stripe unit aligned.
su=value
This is an alternative to using sunit. The su suboption is used to specify the stripe unit for a RAID device or a striped logical volume. The value has to be specified in bytes, (usually using the m or g suffixes). This value must be a multiple of the filesystem block size.
swidth=value
This is used to specify the stripe width for a RAID device or a striped logical volume. The value has to be specified in 512-byte block units. Use the sw suboption to specify the stripe width size in bytes. This suboption is required if -d sunit has been specified and it has to be a multiple of the -d sunit suboption.
sw=value
suboption is an alternative to using swidth. The sw suboption is used to specify the stripe width for a RAID device or striped logical volume. The value is expressed as a multiplier of the stripe unit, usually the same as the number of stripe members in the logical volume configuration, or data disks in a RAID device.
Using su and sw is often easier due to less conversions.
With a 12 drive RAID6 array your stripe width, or sw, is 10. You will need to consult the array controller admin interface and documentation to discover the su value if you don't already know it. Different vendors call this parameter by different names. It could be "chunk size" or "strip size" or other. Some/many vendors don't specify this value at all, giving you only static pre-defined total stripe size options for the array, such as 64KB, 128KB, 1MB, etc, only in power of 2 values. In this case if you have 64KB stripe size and divide by 10 drives in the stripe you end up with a non filesystem block size
multiple: 6553.6 bytes. This presents serious problems for alignment.
In this case you must dig deep to find out exactly how your vendor
controller handles this situation when your effective RAID spindle count is not a power of 2.
So let's assume your vendor does the smart thing and allows you flexibility in specifying per drive strip size. Assume for example the stripe unit (strip, chunk) of the array is 64KB, there are 10 stripe spindles (12-2=10), and the local device name of the LUN is /dev/sdb. To create an aligned XFS filesystem on this you would use something like:
$ mkfs.xfs -d su=64k sw=10 /dev/sdb
When using vendor array hardware that only allows one to define what XFS calls swidth, it is best to use a power of 2 stripe spindle count to get proper alignment. If you use a non power of 2 stripe spindle count the vendor firmware will either round down or round up to create the stripe unit size, and this formula is often not documented.
With such vendor hardware, for a RAID6 array you would want to have 6, 10, or 18 total drives in the array, giving you 4, 8, or 16 stripe spindles. Alternatively, you need to know exactly how the firmware rounds up or down to arrive at the strip block size (sunit).
If you find yourself in such a situation, and are unable to determine the strip size the array firmware is using, you may be better off using the mkfs.xfs defaults, vs guessing and ending up with unaligned writes.
Signature · 6 lines
Stan _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
[ ... ]
Quoted 5 lines
Hi, I have a 4.9 TB iSCSI LUN on a RAID 6 array with twelve 2 TB SATA disks (4.9T is only one of the logical volumes). It will contain several million files of various sizes, but 80% of them will be less than 50 MB. I'm a novice at best and I usually just use the default #mkfs.xfs
/dev/sdx1
The default :-) advice in this list and in the XFS FAQ is that in any recent edition of the XFS tools and XFS code in the kernel the defaults are usually best, unless you have a special situation, for example if the kernel cannot get storage geometry from the storage layer.
Also, "several million" in a about 5,000,000MB filesystem indicates an average file size of 1MB. That's not too small, fortunately. Anyhow consider how long it will take to 'fsck' all that if it gets damaged, or the extra load to backup the whole filetree if backups scan the tree (e.g. RYNC based).
Quoted 6 lines
This is server will be write heavy for about 8 hours a night, but every morning there are many reads to the disk. There is rarely a time where it will be write heavy and read heavy at the same time. Are there other XFS format options that I could use to optimize performance? Any input is greatly appreciated. Thank you.
As usual, the first note is that in general RAID6 is a bad idea, with RMW and reliability (especially during rebuild) issues, but salesmen and management usually love it because it embodies a promise of something for nothing (let's say that the parity RAID industry is the Wall Street of storage system :->).
To mitigate problems In general if you are doing a lot of writing it is very important that the filesystem try to align to address/length of the full RAID stripe, but this should be automatic if the relevant geometry is reported to the Linux kernel. Otherwise thee are many previous messages in this list about that, and the FAQ etc.
Things that you might want to double check in case they matter for you, as to not-'mkfs' options:
* XFS has several limitations on 32b kernels. Just make sure you have a 64b kernel.
* Make really sure your partitions (or LUNs if unpartitioned) are aligned, certainly to a multiple of stripe size, ideally to something larg, at least like 1MiB.
* Recent (let's say at least 2.6.32 or EL57) kernels and editions of XFS tools and partitioning tools (if you use any) are very improved. The newer usually the better.
* Usually just in case explicitly specify at 'mount' (not 'mkfs') time the 'inode64' option; and the 'barrier' option unless you really know better (and pray hard that your storage layer supports it). The 'delaylog' option or its opposite are also something to look carefully into.
* Check carefully whether your app is compatible with the 'noatime' and 'nodiratime' options and enable them if possible, "just in case" :-).
* Look very attentively at the kernel page cache flusher parameters to make it run more often (tom prevent the accumulation of very large gulps of unwritten data) but not too often (to give a chance to the delayed allocator).
As to proper 'mkfs' you may want to look into:
* Explicitly set the sector size because most storage layers lie. In general if possible you should set it to 4096, just in case :-). This also allegedly extends the range where inodes can be stored if you cannot specify 'inode64' at mount time.
* If you have a critically high rate of metadata work (like file creation/deletion, and it seems your case overnight) you may want to ensure that your log is not only aligned, but perhaps on a separate device, and/or you have a host adapter with a large battery backed cache. Logs are small, so it should be easy either way.
* Depending on the degree of multihtreading of your application you may want more/less AGs, but usually on a 4.9TB filetree there will be plenty.
* You may want larger inodes than the default if you have lots of ACLs or your files are written slowly and thus have many extents. They are recommended also for small files but I cannot remember whether XFS really stores small files or directories into the inode (I remember that directories of less than 8 entries are stored in the inode, but I don't know whether depends on its size).
Run first 'mfs.fs -N ....' so it will print out which parameters it will use without actually doing anything.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 8 lines
So let's assume your vendor does the smart thing and allows you flexibility in specifying per drive strip size. Assume for example the stripe unit (strip, chunk) of the array is 64KB, there are 10 stripe spindles (12-2=10), and the local device name of the LUN is
/dev/sdb. To create an aligned XFS filesystem on this you would use something like:$ mkfs.xfs -d su=64k sw=10 /dev/sdb
Great explanations! (some of it I am still trying to understand :-) In this case on my HP P2000 G3, I do have a 64k chunk size so I will do:
$ mkfs.xfs -d su=64k,sw=10 /dev/sdd
Question: Does the above command assume I do not already have a partition created? I was
http://www.fhgfs.com/wiki/wikka.php?wakka=PartitionAlignment reading here that the easiest way to acheive partition alignment is to create the file system directly on the storage device without any paritions - such as $ mkfs.xfs /dev/sdd (and your example above also hints at this)
When I created my current partiton, I used the following commands:
$ parted -a optimal /dev/sdd $ mklabel gpt $ mkpart primary 0 -0 $ q
I would like to align the partiton as well, but I am not sure how to acheive this using parted. This will be the only partition on the LUN, so not sure if I even need to create one (although I do like to stay consistent with my other volumes).
When printing the partition info with parted I see:
# (parted) p
Model: HP P2000 G3 iSCSI (scsi)
Disk /dev/sdd: 4900GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Number Start End Size File system Name Flags 1 1049kB 4900GB 4900GB xfs primary
but from reading, I suspect the Sector size should be more like:
(logical/physical): 512B/65536B. Any thoughts on partition alignment or other thoughts in general? Thank you.
Signature · 7 lines
View this message in context: http://old.nabble.com/Optimal-XFS-formatting-options--tp33140169p33145068.html Sent from the Xfs - General mailing list archive at Nabble.com. _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 7 lines · MikeJeezy
Quoted 6 lines
So let's assume your vendor does the smart thing and allows you flexibility in specifying per drive strip size. Assume for example the stripe unit (strip, chunk) of the array is 64KB, there are 10 stripe spindles (12-2=10), and the local device name of the LUN is
/dev/sdb. To create an aligned XFS filesystem on this you would use something like:
Quoted 6 lines
Quoted 1 line
$ mkfs.xfs -d su=64k sw=10 /dev/sdbGreat explanations! (some of it I am still trying to understand :-) In this case on my HP P2000 G3, I do have a 64k chunk size so I will do:
$ mkfs.xfs -d su=64k,sw=10 /dev/sdd
That should be fine.
Quoted 6 lines
Question: Does the above command assume I do not already have a partition created? I was
http://www.fhgfs.com/wiki/wikka.php?wakka=PartitionAlignment reading here that the easiest way to acheive partition alignment is to create the file system directly on the storage device without any paritions - such as $ mkfs.xfs/dev/sdd(and your example above also hints at this)
That example and command assume you're not using partitions.
Quoted 11 lines
When I created my current partiton, I used the following commands:
$ parted -a optimal /dev/sdd $ mklabel gpt $ mkpart primary 0 -0 $ qI would like to align the partiton as well, but I am not sure how to acheive this using parted. This will be the only partition on the LUN, so not sure if I even need to create one (although I do like to stay consistent with my other volumes).
If your drives have 512 byte physical sectors (not advanced format drives with 4096 byte sectors) then there is no need to worry about partition alignment. And in fact, if you plan to put a single filesystem on this entire 4.9TB virtual drive, you don't need to partition the disk device at all. Recall the dictionary definition of "partition". You're not dividing the whole into smaller pieces here.
Quoted 8 lines
When printing the partition info with parted I see:
# (parted) p
Model: HP P2000 G3 iSCSI (scsi)
Disk/dev/sdd: 4900GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Quoted 5 lines
Number Start End Size File system Name Flags 1 1049kB 4900GB 4900GB xfs primarybut from reading, I suspect the Sector size should be more like:
(logical/physical): 512B/65536B.
No, that 65536 figure is wrong. Their are only two possibilities for sector size (logical/physical): 512/512 and 512/4096. These are the only two disk sector formats currently used on disk drives. Partitioning utils look strictly at disk parameters, not RAID parameters.
Sectors deal with how many books (bytes) fit on each shelf (sector) in the library, and which shelf (sector) we're going to store a given set of books (bytes) on. RAID parameters, such as stripe unit, deal with how many shelves (sectors) worth of books (bytes) we can carry most efficiently down the isle and place on the shelves at one time.
In short, sectors are a destination where we store bytes, much like books on a shelf. A stripe unit acts as a book cart in which we carry a fixed number of books, allowing us to fill a fixed number of shelves most efficiently per cart transported down the isle.
Quoted 2 lines
Any thoughts on partition alignment or
other thoughts in general? Thank you.
Yes, don't use partitions if you don't need to divide your disk device (LUN/virtual disk) into multiple pieces. Now, if you need to make use of snapshots or other volume management features, you may want to create an LVM device on top of the disk device (LUN) and then make your XFS on top of the LVM device. If you have no need for LVM features, I'd say directly format the LUN with XFS, no partition table necessary.
Signature · 6 lines
Stan _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Hi, I have a 4.9 TB iSCSI LUN on a RAID 6 array with twelve 2 TB SATA disks (4.9T is only one of the logical volumes). It will contain several million files of various sizes, but 80% of them will be less than 50 MB. I'm a novice at best and I usually just use the default #mkfs.xfs /dev/sdx1
This is server will be write heavy for about 8 hours a night, but every morning there are many reads to the disk. There is rarely a time where it will be write heavy and read heavy at the same time. Are there other XFS format options that I could use to optimize performance?
Any input is greatly appreciated. Thank you.
Signature · 7 lines
View this message in context: http://old.nabble.com/Optimal-XFS-formatting-options--tp33140169p33140169.html Sent from the Xfs - General mailing list archive at Nabble.com. _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 9 lines · Stan Hoeppner
Quoted 4 lines · MikeJeezy
I would like to align the partiton as well, but I am not sure how to acheive this using parted. This will be the only partition on the LUN, so not sure if I even need to create one (although I do like to stay consistent with my other volumes).
If your drives have 512 byte physical sectors (not advanced format drives with 4096 byte sectors) then there is no need to worry about partition alignment.
That is incorrect. Partitions need to be aligned to the underlying stripe configuration, regardless of the sector size of the drives that make up the stripe. If you do not align the partition to the stripe, then the filesystem will be unaligned no matter how you configure it. Every layer of the storage stack under the filesystem needs to be correctly aligned and sized for filesystem alignment to make any difference to performance.
Quoted 9 lines
Quoted 2 lines
Any thoughts on partition alignment or
other thoughts in general? Thank you.Yes, don't use partitions if you don't need to divide your disk device (LUN/virtual disk) into multiple pieces. Now, if you need to make use of snapshots or other volume management features, you may want to create an LVM device on top of the disk device (LUN) and then make your XFS on top of the LVM device. If you have no need for LVM features, I'd say directly format the LUN with XFS, no partition table necessary.
If you use LVM, then you need to ensure that it is slicing up the device in a manner that is aligned correctly to the underlying stripe, just like if you are using partitions to provide the same functionality. Different technologies, same problem.
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 11 lines · Dave Chinner
Quoted 9 lines · Stan Hoeppner
Quoted 4 lines · MikeJeezy
I would like to align the partiton as well, but I am not sure how to acheive this using parted. This will be the only partition on the LUN, so not sure if I even need to create one (although I do like to stay consistent with my other volumes).
If your drives have 512 byte physical sectors (not advanced format drives with 4096 byte sectors) then there is no need to worry about partition alignment.
Quoted 7 lines
That is incorrect. Partitions need to be aligned to the underlying stripe configuration, regardless of the sector size of the drives that make up the stripe. If you do not align the partition to the stripe, then the filesystem will be unaligned no matter how you configure it. Every layer of the storage stack under the filesystem needs to be correctly aligned and sized for filesystem alignment to make any difference to performance.
Thanks for the correction/reminder Dave. So in this case the first sector of the first partition would need to reside at LBA1280 in this array (655360 byte stripe width, 1280 sectors/stripe), as the partition table itself is going to occupy some sectors at the beginning of the first stripe. By creating the partition at LBA1280 we make sure the first sector of the XFS filesystem is aligned with the first sector of the 2nd stripe.
This exercise demonstrates why it's often preferable to directly format the LUN. If you don't have a _need_ for a partition table, such as cloning/backup software that works at the partition level, or something of that nature, avoid partitions.
Quoted 10 lines
Quoted 9 lines
Quoted 2 lines
Any thoughts on partition alignment or
other thoughts in general? Thank you.Yes, don't use partitions if you don't need to divide your disk device (LUN/virtual disk) into multiple pieces. Now, if you need to make use of snapshots or other volume management features, you may want to create an LVM device on top of the disk device (LUN) and then make your XFS on top of the LVM device. If you have no need for LVM features, I'd say directly format the LUN with XFS, no partition table necessary.
Quoted 4 lines
If you use LVM, then you need to ensure that it is slicing up the device in a manner that is aligned correctly to the underlying stripe, just like if you are using partitions to provide the same functionality. Different technologies, same problem.
If he's doing a single LVM volume then alignment should be automatic during mkfs.xfs shouldn't it?
Signature · 7 lines
Stan _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 7 lines · On Dienstag, 17. Januar 2012 Stan Hoeppner wrote:
Thanks for the correction/reminder Dave. So in this case the first sector of the first partition would need to reside at LBA1280 in this array (655360 byte stripe width, 1280 sectors/stripe), as the partition table itself is going to occupy some sectors at the beginning of the first stripe. By creating the partition at LBA1280 we make sure the first sector of the XFS filesystem is aligned with the first sector of the 2nd stripe.
There's one big problem with that: Many people will sooner or later expand and existing array. If you add one drive, all your nice stripe width alignment becomes bogus, and suddenly your performance will drop.
There's no real way out of that, but three solutions come to my mind:
- backup before expand/restore after expand with new alignment
- leave existing data, just change mount options so after expansion at least new files are going to be aligned to the new stripe width.
- expand array by factors of two. So if you have 10 data drives, add 10 data drives. But that creates other problems (probability of single drive failure + time to recover a single broken disk)
Signature · 6 lines
mit freundlichen Grüssen, Michael Monnerie, Ing. BSc it-management Internet Services: Protéger http://proteger.at [gesprochen: Prot-e-schee] Tel: +43 660 / 415 6531
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Le Tue, 17 Jan 2012 10:19:55 +0100
Michael Monnerie <[email protected]> écrivait:
Quoted 3 lines
- expand array by factors of two. So if you have 10 data drives, add 10 data drives. But that creates other problems (probability of single drive failure + time to recover a single broken disk)
From my experience 20 drives is OK for RAID-6. And rebuild time doesn't change much with array size, anyway.
Misaligned partitions, on the other hand, can easily halve array throughput from my own measurements.
Signature · 6 lines
------------------------------------------------------------------------
Emmanuel Florac | Direction technique
| Intellique
| <[email protected]>
| +33 1 78 94 84 02
------------------------------------------------------------------------_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 9 lines · Michael Monnerie
Quoted 7 lines · On Dienstag, 17. Januar 2012 Stan Hoeppner wrote:
Thanks for the correction/reminder Dave. So in this case the first sector of the first partition would need to reside at LBA1280 in this array (655360 byte stripe width, 1280 sectors/stripe), as the partition table itself is going to occupy some sectors at the beginning of the first stripe. By creating the partition at LBA1280 we make sure the first sector of the XFS filesystem is aligned with the first sector of the 2nd stripe.
Quoted 3 lines
There's one big problem with that: Many people will sooner or later expand and existing array. If you add one drive, all your nice stripe width alignment becomes bogus, and suddenly your performance will drop.
So to be clear, your issue with the above isn't with my partition alignment math WRT the OP's P2000 array, but is with using XFS stripe alignment in general, correct?
Quoted 7 lines
There's no real way out of that, but three solutions come to my mind:
- backup before expand/restore after expand with new alignment
- leave existing data, just change mount options so after expansion at least new files are going to be aligned to the new stripe width.
- expand array by factors of two. So if you have 10 data drives, add 10 data drives. But that creates other problems (probability of single drive failure + time to recover a single broken disk)
There is one really simple way around this issue you describe: don't add drives to an existing array. Simply create another array with new disks, create a new aligned XFS on the array, and mount the filesystem in an appropriate location. There is no 11th Commandment stating one must have a single massive XFS atop all of one's disks. ;)
There is little to no application software today that can't be configured to store its data files across multiple directories. So there's no need to box oneself into the corner you describe above.
Signature · 6 lines
Stan _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 4 lines · Peter Grandi
* XFS has several limitations on 32b kernels. Just make sure you have a 64b kernel.
----
I was unaware that the block size was larger on 64b kernels.
Is that what you are referring to ?
(would be nice)...
One thing I have a Q on -- you (OP), said this was an 'iscsi' box?
That means hookup over an network, right?
You are planning on using a 10Gbit or faster network fabric, right?
a 1Gb ethernet will only get you 125MB/s max... doesn't take much tuning to hit that speed.
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
[ ... ]
Quoted 2 lines
Quoted 2 lines
* XFS has several limitations on 32b kernels. Just make sure you have a 64b kernel.
[ ... ]
Quoted 2 lines
I was unaware that the block size was larger on 64b kernels. Is that what you are referring to ? (would be nice)...
Not as such, the maximum block size is limited by the Linux page cache, that is hw page size, which is for IA32 and AMD64 architectures the same at 4KiB. However other architectures which are natively 64b allow bigger page sizes (notably IA64 [aka Itanium]), so the page cache and thus XFS can do larger blocks sizes.
The limitations of XFS on 32b kernels come from limitations of XFS itself in 32b mode, limitations of Linux in 32b mode, and combined limitations. For example:
* There be 32b inode numbers, which limit inodes to the first 1TB of a filetree if sector size is 512B.
* The 32b block IO subsystems limits partition sizes to 16TiB.
* XFS tools scanning a large filesystem, usually for repair, can run out of the available 32b address space (by default around 2GiB).
Page 5 and 6 here list some limits:
http://oss.sgi.com/projects/xfs/training/xfs_slides_02_overview.pdf
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 3 lines · On Dienstag, 17. Januar 2012 Stan Hoeppner wrote:
So to be clear, your issue with the above isn't with my partition alignment math WRT the OP's P2000 array, but is with using XFS stripe alignment in general, correct?
Yes. I just wanted to document this as people often expand RAIDs and forget to apply the changes to stripe width.
Quoted 10 lines
There is one really simple way around this issue you describe: don't add drives to an existing array. Simply create another array with new disks, create a new aligned XFS on the array, and mount the filesystem in an appropriate location. There is no 11th Commandment stating one must have a single massive XFS atop all of one's disks. ;)
There is little to no application software today that can't be configured to store its data files across multiple directories. So there's no need to box oneself into the corner you describe above.
It's a management burden to do that. I've learned that systems usually are strictly structured in their configuration, so it's often better to extend a RAID and to keep the config, as this is cheaper in the end. At least for the salaries of good admins here in Europe ;-)
Signature · 6 lines
mit freundlichen Grüssen, Michael Monnerie, Ing. BSc it-management Internet Services: Protéger http://proteger.at [gesprochen: Prot-e-schee] Tel: +43 660 / 415 6531
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 4 lines · On Freitag, 20. Januar 2012 Peter Grandi wrote:
* There be 32b inode numbers, which limit inodes to the first 1TB of a filetree if sector size is 512B.
* The 32b block IO subsystems limits partition sizes to 16TiB.
I thought those two have been removed by some updates? I think I remember to have read that. Not that it's too interesting, I've been running on 64b Linux everywhere since AMD has put it in their processors. Should be 10+ years or so.
Signature · 6 lines
mit freundlichen Grüssen, Michael Monnerie, Ing. BSc it-management Internet Services: Protéger http://proteger.at [gesprochen: Prot-e-schee] Tel: +43 660 / 415 6531
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 15 lines · Michael Monnerie
Quoted 3 lines · On Dienstag, 17. Januar 2012 Stan Hoeppner wrote:
So to be clear, your issue with the above isn't with my partition alignment math WRT the OP's P2000 array, but is with using XFS stripe alignment in general, correct?
Yes. I just wanted to document this as people often expand RAIDs and forget to apply the changes to stripe width.
Quoted 7 lines
There is one really simple way around this issue you describe: don't add drives to an existing array. Simply create another array with new disks, create a new aligned XFS on the array, and mount the filesystem in an appropriate location. There is no 11th Commandment stating one must have a single massive XFS atop all of one's disks. ;)
Quoted 8 lines
Quoted 3 lines
There is little to no application software today that can't be configured to store its data files across multiple directories. So there's no need to box oneself into the corner you describe above.
It's a management burden to do that. I've learned that systems usually are strictly structured in their configuration, so it's often better to extend a RAID and to keep the config, as this is cheaper in the end. At least for the salaries of good admins here in Europe ;-)
If ease (or cost) of filesystem administration is of that much greater priority than performance, then why are you using XFS in the first place instead of EXT?
Signature · 6 lines
Stan _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 14 lines · Peter Grandi
[ ... ]
Quoted 2 lines
Quoted 2 lines
* XFS has several limitations on 32b kernels. Just make sure you have a 64b kernel.
[ ... ]
Quoted 2 lines
I was unaware that the block size was larger on 64b kernels. Is that what you are referring to ? (would be nice)...
Not as such, the maximum block size is limited by the Linux page cache, that is hw page size, which is for IA32 and AMD64 architectures the same at 4KiB. However other architectures which are natively 64b allow bigger page sizes (notably IA64 [aka Itanium]), so the page cache and thus XFS can do larger blocks sizes.
Quoted 6 lines
The limitations of XFS on 32b kernels come from limitations of XFS itself in 32b mode, limitations of Linux in 32b mode, and combined limitations. For example:
* There be 32b inode numbers, which limit inodes to the first 1TB of a filetree if sector size is 512B.
Internally XFS still uses 64 bit inode numbers - the on-disk format does not change just because the CPU arch has changed. If you use the stat64() style interfaces, even on 32 bit machines you can access the full 64 bit inode numbers.
Quoted 1 line
* The 32b block IO subsystems limits partition sizes to 16TiB.
The sector_t is a 64 bit number even on 32 bit systems. The problem is that the page cache cannot index past offsets of 16TB. Given that XFS no longer uses the page cache for it's metadata indexing, we could remove this limit in the kernel code if we wanted to. And given that the userpsace tools use direct IO, the page cache limitation doesn't cause problems there, either, because we bypass it.
So in theory we could lift this limit, but there really isn't much demand for >16TB filesystems on 32 bit, because....
Quoted 3 lines
* XFS tools scanning a large filesystem, usually for repair, can run out of the available 32b address space (by default around 2GiB).
.... you need 64 bit systems to handle the userspace memory requirements tools like xfs_check and xfs_repair require to run. If the filesystem is large enough that you can't run repair because it needs more than 2GB of RAM, then you shouldn't be using a 32 bit systems.
Cheers,
Dave.
Signature · 7 lines
Dave Chinner [email protected] _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 3 lines · On Freitag, 20. Januar 2012 Stan Hoeppner wrote:
If ease (or cost) of filesystem administration is of that much greater priority than performance, then why are you using XFS in the first place instead of EXT?
Great experience in recovery of disaster filesystem problems on XFS. A switch to another FS costs a lot of time, and why switch if it works great? And administration comes down to mkfs, mount, maybe xfs_fsr, in disaster xfs_repair, and sometimes xfs_growfs. Basically nothing.
Also, this list has been of great help during the years, whenever there were problems they got fixed. That's ease of administration :-)
Signature · 6 lines
mit freundlichen Grüssen, Michael Monnerie, Ing. BSc it-management Internet Services: Protéger http://proteger.at [gesprochen: Prot-e-schee] Tel: +43 660 / 415 6531
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs