XFS and 3.2TB Partition
I am trying to create a 3.2TB partition on my Raid 5. Is there a document that could help?
I have a 3ware 9500 controller and 8 *500GB sata drives configured into a single RAID 5 array.
I am running linux 2.6.16 with the 3ware drivers compiled into the kernel.
I've tried a couple of different means to create the partition and format the file system with xfs without success (or confidence that I haven't done something wrong).
1. FDISK
I've tried fdisk on the array to create the partition; but it forces me enter the number of cylinders before letting me create the partition. I enter the largest number of cylinders since I'm not sure how to calculate the correct cylinder number across an 8 disk RAID 5 array.
I then create the partition starting at 0 (or whatever the default was) and ending at 3500GB.
Once the partition is created this way, I can mkfs.xfs; but I'm a little hesitant to use this since I input and arbitrary cylinder number.
Thoughts on what to use for the correct cylinder count with fdisk?
2. PARTED
I've tried to use parted without any success. Here is what I've tried and the errors I get.
Quoted 1 line
parted
parted> mklabel gpt
parted> mkpart primary 0 3500GB
parted> quit
ok - the partition now exists. If I use ext2 everything works ok.
however, when I run
Quoted 1 line
mkfs.xfs
/dev/sda1
the file system is formated but is truncated to to 2TB.
Any advice/pointers on how to partition and format a 3.2TB raid 5 array would be much appreciated.
Thanks,
Chris....
Quoted 9 lines · Chris Hane
2. PARTED
I've tried to use parted without any success. Here is what I've tried and the errors I get.
Quoted 1 line
parted
parted> mklabel gpt
parted> mkpart primary 0 3500GB
parted> quit
Quoted 7 lines
ok - the partition now exists. If I use ext2 everything works ok.
however, when I run
Quoted 1 line
mkfs.xfs
/dev/sda1the file system is formated but is truncated to to 2TB.
I'd think parted should be able to handle large devices...
mkfs.xfs uses the BLKGETSIZE64 and BLKGETSIZE ioctls to determine device size; you might either instrument xfsprogs/libxfs/linux.c and re-run, or strace your mkfs command looking for those ioctl calls, to see what size it's getting back from the kernel.
-Eric
Hi Chris,
Quoted 2 lines
I am trying to create a 3.2TB partition on my Raid 5. Is there a document that could help?
You can get generic information on partitions and how to create them at
http://www.faqs.org/docs/Linux-mini/Partition.html
Quoted 7 lines
I have a 3ware 9500 controller and 8 *500GB sata drives configured into a single RAID 5 array. I am running linux 2.6.16 with the 3ware drivers compiled into the kernel.
I've tried a couple of different means to create the partition and format the file system with xfs without success (or confidence that I haven't done something wrong).
1. FDISK
Quoted 8 lines
I've tried fdisk on the array to create the partition; but it forces me enter the number of cylinders before letting me create the partition. I enter the largest number of cylinders since I'm not sure how to calculate the correct cylinder number across an 8 disk RAID 5 array.
I then create the partition starting at 0 (or whatever the default was) and ending at 3500GB.
Once the partition is created this way, I can mkfs.xfs; but I'm a little hesitant to use this since I input and arbitrary cylinder number.
Thoughts on what to use for the correct cylinder count with fdisk?
It is not actually arbitrary. There are various options to create a partition - can specify the size, sectors or cylinders. fdisk -l /dev/<device> can hopefully give you details about the size of the disk, cylinders etc. Now, it is upto to you as to how many partiotions you want to create and use.
Quoted 9 lines
2. PARTED
I've tried to use parted without any success. Here is what I've tried and the errors I get.
Quoted 1 line
parted
parted> mklabel gpt
parted> mkpart primary 0 3500GB
parted> quit
Quoted 7 lines
ok - the partition now exists. If I use ext2 everything works ok.
however, when I run
Quoted 1 line
mkfs.xfs
/dev/sda1the file system is formated but is truncated to to 2TB.
XFS can limit the FS size this way when AG count is just 2. xfs_info <mnt_point> will list the details about the number of AG count. XFS can support about 1 TB in one AG. By default, it calculates the agcount automatically unless you force them. Your command line option does not suggest this.
The 2 TB limit might be coming, perhaps, because parted uses 4 byte addresses data structures. This way, it can support maximum of 2^32 sectors of 512 size ~ 41 bits ~ 2TB. It is my guess, though. -shailendra
Quoted 5 lines · Chris Hane
I have a 3ware 9500 controller and 8 *500GB sata drives configured into
a single RAID 5 array.
You might want to google around a bit and verify that your controller supports
lun's over 2TB.
Am Donnerstag, 31. August 2006 18:57 schrieben Sie:
Quoted 6 lines
I am trying to create a 3.2TB partition on my Raid 5. Is there a document that could help?
I have a 3ware 9500 controller and 8 *500GB sata drives configured into a single RAID 5 array.
I have a Raid with about 5TB and no problems creating an xfs filesystem on it. The system is Novell SLES10 with a 2.6.16.21 kernel.
At first there was a problem with the raid. The firmware of the raid device needed an upgrade. Bevore the upgrade I had a maximum of 2TB.
In dmesg (or /var/log/boot.msg on SLES10) you should see something like this message if the device (sdc here) is handled correctly:
<5>sdc : very big device. try to use READ CAPACITY(16). <5>SCSI device sdc: 10156243968 512-byte hdwr sectors (5199997 MB) <5>sdc: Write Protect is off
<7>sdc: Mode Sense: cb 00 00 08
<5>SCSI device sdc: drive cache: write back
<5>sdc : very big device. try to use READ CAPACITY(16). <5>SCSI device sdc: 10156243968 512-byte hdwr sectors (5199997 MB)
Bevore the firmware update there was an error when trying to read the capacity via READ CAPACITY(16).
I created the partitiions using parted. fdisk did not work.
Have a nice day
Rainer
Signature · 6 lines
--------------------------------------------------------------------------- Rainer Krienke, Universitaet Koblenz, Rechenzentrum, Raum A022 Universitaetsstrasse 1, 56070 Koblenz, Tel: +49 261287 -1312, Fax: -1001312 Mail: [email protected], Web: http://www.uni-koblenz.de/~krienke Get my public PGP key: http://www.uni-koblenz.de/~krienke/mypgp.html ---------------------------------------------------------------------------
I had that problem when I was using an adaptec aic79xx adapter, using a new LSI-Logic one fixed the problem for me. Could it be a limit with your controller/controller driver?
Quoted 44 lines · Chris Hane
I am trying to create a 3.2TB partition on my Raid 5. Is there a document that could help?
I have a 3ware 9500 controller and 8 *500GB sata drives configured into a single RAID 5 array.
I am running linux 2.6.16 with the 3ware drivers compiled into the kernel.
I've tried a couple of different means to create the partition and format the file system with xfs without success (or confidence that I haven't done something wrong).
1. FDISK
I've tried fdisk on the array to create the partition; but it forces me enter the number of cylinders before letting me create the partition. I enter the largest number of cylinders since I'm not sure how to calculate the correct cylinder number across an 8 disk RAID 5 array.
I then create the partition starting at 0 (or whatever the default was) and ending at 3500GB.
Once the partition is created this way, I can mkfs.xfs; but I'm a little hesitant to use this since I input and arbitrary cylinder number.
Thoughts on what to use for the correct cylinder count with fdisk?
2. PARTED
I've tried to use parted without any success. Here is what I've tried and the errors I get.
Quoted 1 line
parted
parted> mklabel gpt
parted> mkpart primary 0 3500GB
parted> quitok - the partition now exists. If I use ext2 everything works ok.
however, when I run
Quoted 1 line
mkfs.xfs
/dev/sda1the file system is formated but is truncated to to 2TB.
Any advice/pointers on how to partition and format a 3.2TB raid 5 array would be much appreciated.
Thanks,
Chris....
Signature · 5 lines
[email protected] Linux Systems Administrator Double Negative tel: +44 (0)20 7534 4400 fax: +44 (0)20 7534 4452 77 shaftesbury avenue, w1d 5du, London
Thank for the input. I appreciate everyones help!
I believe I am going to end up not partitioning the raid array and using it directly (as described in an email which I copied below).
When we get our next large storage machine in (we've are going to need a couple over the next year to store CD & DVD ISO Images), I'm going to experiment some more with the suggestions everyone has given me here.
As an FYI: I'm using the latest versions of everything (parted 1.7.1, kernel 2.6.16, 3ware raid controller brand new)
Thanks for the help,
Chris....
Quoted 28 lines · Peter Grandi
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
On Thu, 31 Aug 2006 12:57:22 -0400, Chris Hane
<[email protected]> said:chrishane> I am trying to create a 3.2TB partition on my Raid 5. chrishane> Is there a document that could help?
The 9500 is fairly recent, so it should not have a lot of 2TB problems. But there are 2TB limits in several places. For example old versions of the Linux kernel don't support more than 2TB per _filesystem_.
But I suspect that you are trying to create partitions in the sense of the MS-DOS/MS-Windows partitioning scheme. Check carefully whether that partitioning scheme allos partitions larger than 2TB :-).
Anyhow, usually for very large filesystems you don't need partitions at all. Just use '
/dev/sda'. Or check the other partitioning schemes supported by Linux, some may have higher limits.chrishane> I have a 3ware 9500 controller and 8 *500GB sata chrishane> drives configured into a single RAID 5 array.
Using RAID5 with 8 drives is a great crime. Nothing to do with your partitioning problems, but since you mentioned it... Consider reading carefully <URL:http://WWW.BAARF.com/>
Quoted 28 lines · Rainer Krienke
Am Donnerstag, 31. August 2006 18:57 schrieben Sie:
Quoted 6 lines
I am trying to create a 3.2TB partition on my Raid 5. Is there a document that could help?
I have a 3ware 9500 controller and 8 *500GB sata drives configured into a single RAID 5 array.
I have a Raid with about 5TB and no problems creating an xfs filesystem on it. The system is Novell SLES10 with a 2.6.16.21 kernel.
At first there was a problem with the raid. The firmware of the raid device needed an upgrade. Bevore the upgrade I had a maximum of 2TB.
In dmesg (or
/var/log/boot.msgon SLES10) you should see something like this message if the device (sdc here) is handled correctly:<5>sdc : very big device. try to use READ CAPACITY(16). <5>SCSI device sdc: 10156243968 512-byte hdwr sectors (5199997 MB) <5>sdc: Write Protect is off
<7>sdc: Mode Sense: cb 00 00 08
<5>SCSI device sdc: drive cache: write back
<5>sdc : very big device. try to use READ CAPACITY(16). <5>SCSI device sdc: 10156243968 512-byte hdwr sectors (5199997 MB)Bevore the firmware update there was an error when trying to read the capacity via READ CAPACITY(16).
I created the partitiions using parted. fdisk did not work.
Have a nice day
Rainer
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
On Thu, 31 Aug 2006 12:57:22 -0400, Chris Hane
<[email protected]> said:
chrishane> I've tried to use parted without any success. Here chrishane> is what I've tried and the errors I get.
chrishane> parted
parted> mklabel gpt
parted> mkpart primary 0 3500GB
parted> quit
Uhm, I had missed the 'mklabel gpt'. Unlike 'fdisk', which only supports DOS-style partitions with a 2^32 sector limit (2TiB indeed) the fairly recent GPT partition scheme (disk labeling in the Linux kernel jargon) should support just about any size partitions, with a 2^64 sector limit.
It is somewhat worrying that this happens:
chrishane> ok - the partition now exists. If I use ext2 chrishane> everything works ok. however, when I run mkfs.xfs chrishane> /dev/sda1 the file system is formated but is chrishane> truncated to to 2TB.
Now, you have succeeded in creating a 3.2TiB partition with 'ext2' on '/dev/sda1' with GPT, and with XFS directly on '/dev/sda'. It is rather strange that it apparently did not work with XFS on '/dev/sda1' with GPT.
perhaps 'mkfs.xfs' somehow has problems with the GPT scheme, but that is indeed strange. It would have been interesting to see what 'grep /dev/sda /proc/partitions' would have returned.
It might have been interesting to follow this suggestion too
sandeen> mkfs.xfs uses the BLKGETSIZE64 and BLKGETSIZE ioctls sandeen> to determine device size; you might either instrument sandeen> xfsprogs/libxfs/linux.c and re-run, or strace your sandeen> mkfs command looking for those ioctl calls, to see sandeen> what size it's getting back from the kernel.
to see why 'mkfs.ext2' seems to get the right size of a GPT partition but apparently not 'mkfs.xfs'.
BTW, I hope that you have got a 64 bit system and lots of RAM for that 3.2TiB filesystem...
http://OSS.SGI.com/archives/linux-xfs/2005-08/msg00045.html
BTW, for those following this thread: what other ways to create and manage GPT disk labels are available under GNU/Linux other than 'parted'?