sw and su for hardware RAID10 (w/ LVM)
RHEL6.x + XFS that comes w/ Red Hat's scalable file system add on. We have two PowerVault MD3260e's each configured with a 30 disk RAID10 (15 RAID groups) exposed to our server. Segment size is 128K (in Dell's world I'm not sure if this means my stripe width is 128K*15?)
Have set up a concatenated LVM volume on top of these two "virtual disks" (with lvcreate -i 2).
By default LVM says it's used a stripe width of 64K.
# lvs -o path,size,stripes,stripe_size Path LSize #Str Stripe /dev/agsfac_vg00/lv00 100.00t 2 64.00k
Unsure if these defaults should be adjusted.
I'm trying to figure out the appropriate sw/su values to use per:
Am considering either just going with defaults (XFS should pull from LVM I think) or doing something like sw=2,su=128K. However, maybe I should be doing sw=2,su=1920K? And perhaps my LVM stripe width should be adjusted?
Thanks,
Ray
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 4 lines · Ray Van Dolson
RHEL6.x + XFS that comes w/ Red Hat's scalable file system add on. We have two PowerVault MD3260e's each configured with a 30 disk RAID10 (15 RAID groups) exposed to our server. Segment size is 128K (in Dell's world I'm not sure if this means my stripe width is 128K*15?)
128KB must be the stripe unit.
Quoted 2 lines
Have set up a concatenated LVM volume on top of these two "virtual disks" (with lvcreate -i 2).
This is because you created a 2 stripe array, not a concatenation.
Quoted 5 lines
By default LVM says it's used a stripe width of 64K.
# lvs -o path,size,stripes,stripe_size Path LSize #Str Stripe /dev/agsfac_vg00/lv00 100.00t 2 64.00k
from lvcreate(8)
-i, --stripes Stripes
Gives the number of stripes...
Quoted 10 lines
Unsure if these defaults should be adjusted.
I'm trying to figure out the appropriate sw/su values to use per:
Am considering either just going with defaults (XFS should pull from LVM I think) or doing something like sw=2,su=128K. However, maybe I should be doing sw=2,su=1920K? And perhaps my LVM stripe width should be adjusted?
Why don't you first tell us what you want? You say at the top that you created a concatenation, but at the bottom you say LVM stripe. So first tell us which one you actually want, because the XFS alignment is radically different for each.
Then tell us why you must use LVM instead of md. md has fewer problems/limitations for stripes and concat than LVM, and is much easier to configure.
Signature · 6 lines
Stan _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs
Quoted 35 lines · Stan Hoeppner
Quoted 4 lines · Ray Van Dolson
RHEL6.x + XFS that comes w/ Red Hat's scalable file system add on. We have two PowerVault MD3260e's each configured with a 30 disk RAID10 (15 RAID groups) exposed to our server. Segment size is 128K (in Dell's world I'm not sure if this means my stripe width is 128K*15?)
128KB must be the stripe unit.
Quoted 2 lines
Have set up a concatenated LVM volume on top of these two "virtual disks" (with lvcreate -i 2).
This is because you created a 2 stripe array, not a concatenation.
Quoted 5 lines
By default LVM says it's used a stripe width of 64K.
# lvs -o path,size,stripes,stripe_size Path LSize #Str Stripe /dev/agsfac_vg00/lv00 100.00t 2 64.00kfrom lvcreate(8)
-i, --stripes Stripes
Gives the number of stripes...Quoted 10 lines
Unsure if these defaults should be adjusted.
I'm trying to figure out the appropriate sw/su values to use per:
Am considering either just going with defaults (XFS should pull from LVM I think) or doing something like sw=2,su=128K. However, maybe I should be doing sw=2,su=1920K? And perhaps my LVM stripe width should be adjusted?
Quoted 8 lines
Why don't you first tell us what you want? You say at the top that you created a concatenation, but at the bottom you say LVM stripe. So first tell us which one you actually want, because the XFS alignment is radically different for each.
Then tell us why you must use LVM instead of md. md has fewer problems/limitations for stripes and concat than LVM, and is much easier to configure.
Yes, misused the term concatenation. Striping is what I'm afer (want to use all of my LUNs equally).
I don't know that I necessarily need to use LVM here. No need for snapshots, just after the best "performance" for multiple NAS sourced (via Samba) sequential write or read streams (but not read/write at the same time).
My setup is as follows right now:
MD3260_1 -> Disk Group 0 (RAID10 - 15 RG's, 128K segment size) -> 2
Virtual Disks (one per controller)MD3260_2 -> Disk Group 0 (RAID10 - 15 RG's, 128K segment size) -> 2
Virtual Disks (one per controller)
So I see four equally sized LUNs on my RHEL box, each with one active path and one passive path (using Linux MPIO).
I'll set up a striped md array across these four LUNs using a 128K chunk size.
Things work pretty well with the xfs default, so may stick with that, but to try and get it as "right" as possible, I'm thinking I should be using a su=128k value, but am not sure on the sw value. It's either:
- 4 (four LUNs as far as my OS is concerned)
- 30 (15 RAID groups per MD3260)
I'm thinking probably 4 is the right answer since the RAID groups on my PowerVaults are all abstracted.
Ray
_______________________________________________
xfs mailing list
[email protected]
http://oss.sgi.com/mailman/listinfo/xfs
Quoted 41 lines · Ray Van Dolson
Quoted 40 lines · Stan Hoeppner
Quoted 4 lines · Ray Van Dolson
RHEL6.x + XFS that comes w/ Red Hat's scalable file system add on. We have two PowerVault MD3260e's each configured with a 30 disk RAID10 (15 RAID groups) exposed to our server. Segment size is 128K (in Dell's world I'm not sure if this means my stripe width is 128K*15?)
128KB must be the stripe unit.
Quoted 2 lines
Have set up a concatenated LVM volume on top of these two "virtual disks" (with lvcreate -i 2).
This is because you created a 2 stripe array, not a concatenation.
Quoted 5 lines
By default LVM says it's used a stripe width of 64K.
# lvs -o path,size,stripes,stripe_size Path LSize #Str Stripe /dev/agsfac_vg00/lv00 100.00t 2 64.00kfrom lvcreate(8)
-i, --stripes Stripes
Gives the number of stripes...Quoted 10 lines
Unsure if these defaults should be adjusted.
I'm trying to figure out the appropriate sw/su values to use per:
Am considering either just going with defaults (XFS should pull from LVM I think) or doing something like sw=2,su=128K. However, maybe I should be doing sw=2,su=1920K? And perhaps my LVM stripe width should be adjusted?
Why don't you first tell us what you want? You say at the top that you created a concatenation, but at the bottom you say LVM stripe. So first tell us which one you actually want, because the XFS alignment is radically different for each.
Quoted 6 lines
Quoted 3 lines
Then tell us why you must use LVM instead of md. md has fewer problems/limitations for stripes and concat than LVM, and is much easier to configure.
Yes, misused the term concatenation. Striping is what I'm afer (want to use all of my LUNs equally).
Striping does not guarantee equal distribution of data.
Quoted 6 lines
I don't know that I necessarily need to use LVM here. No need for snapshots, just after the best "performance" for multiple NAS sourced (via Samba) sequential write or read streams (but not read/write at the same time).
My setup is as follows right now:
You need to figure out exactly what you have because none of what you state below makes any sense whatsoever.
Quoted 4 lines
MD3260_1-> Disk Group 0 (RAID10 - 15 RG's, 128K segment size) -> 2
Virtual Disks (one per controller)MD3260_2-> Disk Group 0 (RAID10 - 15 RG's, 128K segment size) -> 2
Virtual Disks (one per controller)
This doesn't tell us how many disks are in each RAID10. That is required information.
It does not show the relationship between a "Virtual Disk" and a RAID10 array.
Quoted 2 lines
So I see four equally sized LUNs on my RHEL box, each with one active path and one passive path (using Linux MPIO).
This does not tell us the size of each LUN. That is required information. The number and size of the RAID10 arrays is also required.
Quoted 2 lines
I'll set up a striped md array across these four LUNs using a 128K chunk size.
This is wrong, no matter what the actual numbers are for everything up above, because the stripe unit of a nested stripe is always equal to the stripe width of the constituent arrays it stitches together.
Quoted 3 lines
Things work pretty well with the xfs default, so may stick with that, but to try and get it as "right" as possible, I'm thinking I should be using a su=128k value, but am not sure on the sw value. It's either:
You only align XFS if you have an allocation heavy workload where the file sizes are smaller but very close to stripe width, or many times larger than stripe width. You *appear* to currently have a 3.84 MB stripe width. That is yet to be confirmed by you.
Quoted 1 line
- 4 (four LUNs as far as my OS is concerned)
My guess is that 2 of the 4 point to the same RAID10, both to the entire capacity, or to half the capacity. If you have 2 carvings of each RAID10 that are called "Virtual Disks" which are each mapped out a different controller as a distinct LUN, then this is completely wrong for what you want to accomplish.
Quoted 1 line
- 30 (15 RAID groups per MD3260)
So Dell calls the 15 RAID1 mirror of a RAID10 "RAID groups"? What does it call the RAID10 itself?
Quoted 2 lines
I'm thinking probably 4 is the right answer since the RAID groups on my PowerVaults are all abstracted.
If you are unable to figure out exactly how your RAIDs, VGs, and LUNs are configured, do not under any circumstances align XFS. All it will do is decrease performance. The default 4KB alignment is far, far better than misalignment.
Signature · 6 lines
Stan _______________________________________________ xfs mailing list [email protected] http://oss.sgi.com/mailman/listinfo/xfs