LVM2
I recently saw Alan Cox say on this list that LVM won't handle more than 2 terabytes. Is this LVM2 or LVM? What is the maximum amount of disk space LVM2 (or any other RAID/MIRROR capable technology that is in Linus's kernel) handle? I am talking with various people and we are looking at Samba on Linux to do several different namespaces (obviously one tree), most averaging about 3 terabytes, but one would have in excess of 20 terabytes. We are looking at using 320 to 500 gigabyte drives in these arrays. (How? IEEE-1394. Which brings a question I will ask in a second email.)
Is RAID 5 all that bad using this software method? Is RAID 5 available?
Trever Adams
--
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -- Benjamin Franklin, 1759
A logical volume in LVM will not handle more than 2TB. You can tie together the LVs in a volume group, thus going over the 2TB limit. Choose your filesystem well though, some have a 2TB limit too.
Disk size: What are you doing with it. 500GB disks are ATA (maybe SATA). ATA is good for low end servers or near line storage, SATA can be used equally to SCSI (I am going to suffer for this remark).
RAID5 in software works pretty good (survived a failed disk, and recovered another failing raid in 1 month). Hardware is better since you don't have a boot partition left which is usually just present on one disk (you can mirror that yourself ofcourse).
Regards,
Norbert van Nobelen
Quoted 22 lines · you
I recently saw Alan Cox say on this list that LVM won't handle more than 2 terabytes. Is this LVM2 or LVM? What is the maximum amount of disk space LVM2 (or any other RAID/MIRROR capable technology that is in Linus's kernel) handle? I am talking with various people and we are looking at Samba on Linux to do several different namespaces (obviously one tree), most averaging about 3 terabytes, but one would have in excess of 20 terabytes. We are looking at using 320 to 500 gigabyte drives in these arrays. (How? IEEE-1394. Which brings a question I will ask in a second email.)
Is RAID 5 all that bad using this software method? Is RAID 5 available?
Trever Adams
--
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -- Benjamin Franklin, 1759-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 2 lines · Norbert van Nobelen
A logical volume in LVM will not handle more than 2TB. You can tie together the LVs in a volume group, thus going over the 2TB limit.
Confused over terminology?
Tie PVs together to form a VG, then divide VG up into LVs.
Size limit depends on metadata format and the kernel: old LVM1 format has lower size limits - see the vgcreate man page.
New LVM2 metadata format relaxes those limits and lets you have LVs > 2TB with a 2.6 kernel.
Alasdair
Signature · 1 line
[email protected]
PV = the device
VG = groups of them (the RAID5 array?)
LV = what? the file system?
So, from what you are telling me, and the man page, 2.6.x with LVM2 can have basically any size of PV, VG, and LV I want.
Am I flawed in my understanding?
Thank you,
Trever
Quoted 7 lines · Alasdair G Kergon
Quoted 2 lines · Norbert van Nobelen
A logical volume in LVM will not handle more than 2TB. You can tie together the LVs in a volume group, thus going over the 2TB limit.
Confused over terminology?
Tie PVs together to form a VG, then divide VG up into LVs.
Quoted 7 lines
Size limit depends on metadata format and the kernel: old LVM1 format has lower size limits - see the vgcreate man page.
New LVM2 metadata format relaxes those limits and lets you have LVs > 2TB with a 2.6 kernel.
Alasdair
--
"Assassination is the extreme form of censorship." -- George Bernard Shaw (1856-1950)
Quoted 3 lines · Trever L. Adams
PV = the device
VG = groups of them (the RAID5 array?)
LV = what? the file system?
http://www.tldp.org/HOWTO/LVM-HOWTO/anatomy.html
http://www.novell.com/products/linuxenterpriseserver8/whitepapers/LVM.pdf
[Out-of-date now, but descriptions of concepts still useful.]
LVM mailing list for LVM questions:
https://www.redhat.com/mailman/listinfo/linux-lvm
Alasdair
Signature · 1 line
[email protected]
Even as LVM user, guess what I used before answering (-:
Quoted 13 lines · Alasdair G Kergon
Quoted 3 lines · Trever L. Adams
PV = the device
VG = groups of them (the RAID5 array?)
LV = what? the file system?http://www.tldp.org/HOWTO/LVM-HOWTO/anatomy.html
http://www.novell.com/products/linuxenterpriseserver8/whitepapers/LVM.pdf
[Out-of-date now, but descriptions of concepts still useful.]LVM mailing list for LVM questions:
https://www.redhat.com/mailman/listinfo/linux-lvmAlasdair
It is for a group. For the most part it is data access/retention. Writes and such would be more similar to a desktop. I would use SATA if they were (nearly) equally priced and there were awesome 1394 to SATA bridge chips that worked well with Linux. So, right now, I am looking at ATA to 1394.
So, to get 2TB of RAID5 you have 6 500 GB disks right? So, will this work within on LV? Or is it 2TB of diskspace total? So, are volume groups pretty fault tolerant if you have a bunch of RAID5 LVs below them? This is my one worry about this.
Second, you mentioned file systems. We were talking about ext3. I have never used any others in Linux (barring ext2, minixfs, and fat). I had heard XFS from IBM was pretty good. I would rather not use reiserfs.
Any recommendations.
Trever
P.S. Why won't an LV support over 2TB?
S.P.S. I am not really worried about the boot and programs drive. They will be spun down most of the time I am sure.
Quoted 40 lines · Norbert van Nobelen
A logical volume in LVM will not handle more than 2TB. You can tie together the LVs in a volume group, thus going over the 2TB limit. Choose your filesystem well though, some have a 2TB limit too.
Disk size: What are you doing with it. 500GB disks are ATA (maybe SATA). ATA is good for low end servers or near line storage, SATA can be used equally to SCSI (I am going to suffer for this remark).
RAID5 in software works pretty good (survived a failed disk, and recovered another failing raid in 1 month). Hardware is better since you don't have a boot partition left which is usually just present on one disk (you can mirror that yourself ofcourse).
Regards,
Norbert van Nobelen
Quoted 17 lines · you
I recently saw Alan Cox say on this list that LVM won't handle more than 2 terabytes. Is this LVM2 or LVM? What is the maximum amount of disk space LVM2 (or any other RAID/MIRROR capable technology that is in Linus's kernel) handle? I am talking with various people and we are looking at Samba on Linux to do several different namespaces (obviously one tree), most averaging about 3 terabytes, but one would have in excess of 20 terabytes. We are looking at using 320 to 500 gigabyte drives in these arrays. (How? IEEE-1394. Which brings a question I will ask in a second email.)
Is RAID 5 all that bad using this software method? Is RAID 5 available?
Trever Adams
--
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -- Benjamin Franklin, 1759Quoted 5 lines
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 6 lines
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
--
"Assassination is the extreme form of censorship." -- George Bernard Shaw (1856-1950)
Quoted 3 lines · Trever L. Adams
Second, you mentioned file systems. We were talking about ext3. I have never used any others in Linux (barring ext2, minixfs, and fat). I had heard XFS from IBM was pretty good. I would rather not use reiserfs.
XFS is from SGI. JFS is from IBM.
-- wli
XFS is an SGI project.
http://oss.sgi.com/
I've been using it for quite a while and am quite happy with it; it is very fast and very fault tolerant. The only warning I'd like to give about it is it seems that some Linux developers seem to have a bad taste in their mouth when it comes to XFS; go figure.
Signature · 1 line
jeffrey hundstad
Quoted 83 lines · Trever L. Adams
It is for a group. For the most part it is data access/retention. Writes and such would be more similar to a desktop. I would use SATA if they were (nearly) equally priced and there were awesome 1394 to SATA bridge chips that worked well with Linux. So, right now, I am looking at ATA to 1394.
So, to get 2TB of RAID5 you have 6 500 GB disks right? So, will this work within on LV? Or is it 2TB of diskspace total? So, are volume groups pretty fault tolerant if you have a bunch of RAID5 LVs below them? This is my one worry about this.
Second, you mentioned file systems. We were talking about ext3. I have never used any others in Linux (barring ext2, minixfs, and fat). I had heard XFS from IBM was pretty good. I would rather not use reiserfs.
Any recommendations.
Trever
P.S. Why won't an LV support over 2TB?
S.P.S. I am not really worried about the boot and programs drive. They will be spun down most of the time I am sure.Quoted 37 lines · Norbert van Nobelen
A logical volume in LVM will not handle more than 2TB. You can tie together the LVs in a volume group, thus going over the 2TB limit. Choose your filesystem well though, some have a 2TB limit too.
Disk size: What are you doing with it. 500GB disks are ATA (maybe SATA). ATA is good for low end servers or near line storage, SATA can be used equally to SCSI (I am going to suffer for this remark).
RAID5 in software works pretty good (survived a failed disk, and recovered another failing raid in 1 month). Hardware is better since you don't have a boot partition left which is usually just present on one disk (you can mirror that yourself ofcourse).
Regards,
Norbert van Nobelen
Quoted 17 lines · you
I recently saw Alan Cox say on this list that LVM won't handle more than 2 terabytes. Is this LVM2 or LVM? What is the maximum amount of disk space LVM2 (or any other RAID/MIRROR capable technology that is in Linus's kernel) handle? I am talking with various people and we are looking at Samba on Linux to do several different namespaces (obviously one tree), most averaging about 3 terabytes, but one would have in excess of 20 terabytes. We are looking at using 320 to 500 gigabyte drives in these arrays. (How? IEEE-1394. Which brings a question I will ask in a second email.)
Is RAID 5 all that bad using this software method? Is RAID 5 available?
Trever Adams
--
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -- Benjamin Franklin, 1759Quoted 7 lines
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/Quoted 8 lines
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/--
"Assassination is the extreme form of censorship." -- George Bernard Shaw (1856-1950)-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 11 lines · Trever L. Adams
It is for a group. For the most part it is data access/retention. Writes and such would be more similar to a desktop. I would use SATA if they were (nearly) equally priced and there were awesome 1394 to SATA bridge chips that worked well with Linux. So, right now, I am looking at ATA to 1394.
So, to get 2TB of RAID5 you have 6 500 GB disks right? So, will this work within on LV? Or is it 2TB of diskspace total? So, are volume groups pretty fault tolerant if you have a bunch of RAID5 LVs below them? This is my one worry about this.
Quoted 8 lines
Second, you mentioned file systems. We were talking about ext3. I have never used any others in Linux (barring ext2, minixfs, and fat). I had heard XFS from IBM was pretty good. I would rather not use reiserfs.
Any recommendations.
Trever
They all forgot to mention one more limitation, the maximum filesystem size supported by the address_space structure in linux. If you are running on ia32, then you get stuck with 2^32 filesystem blocks, or 16 Tbytes in one filesystem because of the way an address space structure is used to cache the metadata. If you use an Athlon 64 that limitation goes away.
Steve
With using RAID5 you can choose yourself howmany hot standby/failover disks you want to use. The number (or ratio) of disks used for failover of your raid will determine the chance that you have when one disk fails and complete failure of a raid.
It is still pretty safe just to have 7 active disks and 1 hot standby disk, thus getting you 3500 Manufacturers GB (3.34TB)
About LVM on a RAID:
With hardware RAID it will work for sure.
With software RAID:
Make the RAID partitions and build your RAID.
On that RAID, set the partition type to LVM
And it should work
On LVM you can then use ext3 (your preferred fs)
Quoted 75 lines · you
It is for a group. For the most part it is data access/retention. Writes and such would be more similar to a desktop. I would use SATA if they were (nearly) equally priced and there were awesome 1394 to SATA bridge chips that worked well with Linux. So, right now, I am looking at ATA to 1394.
So, to get 2TB of RAID5 you have 6 500 GB disks right? So, will this work within on LV? Or is it 2TB of diskspace total? So, are volume groups pretty fault tolerant if you have a bunch of RAID5 LVs below them? This is my one worry about this.
Second, you mentioned file systems. We were talking about ext3. I have never used any others in Linux (barring ext2, minixfs, and fat). I had heard XFS from IBM was pretty good. I would rather not use reiserfs.
Any recommendations.
Trever
P.S. Why won't an LV support over 2TB?
S.P.S. I am not really worried about the boot and programs drive. They will be spun down most of the time I am sure.Quoted 41 lines · Norbert van Nobelen
A logical volume in LVM will not handle more than 2TB. You can tie together the LVs in a volume group, thus going over the 2TB limit. Choose your filesystem well though, some have a 2TB limit too.
Disk size: What are you doing with it. 500GB disks are ATA (maybe SATA). ATA is good for low end servers or near line storage, SATA can be used equally to SCSI (I am going to suffer for this remark).
RAID5 in software works pretty good (survived a failed disk, and recovered another failing raid in 1 month). Hardware is better since you don't have a boot partition left which is usually just present on one disk (you can mirror that yourself ofcourse).
Regards,
Norbert van Nobelen
Quoted 17 lines · you
I recently saw Alan Cox say on this list that LVM won't handle more than 2 terabytes. Is this LVM2 or LVM? What is the maximum amount of disk space LVM2 (or any other RAID/MIRROR capable technology that is in Linus's kernel) handle? I am talking with various people and we are looking at Samba on Linux to do several different namespaces (obviously one tree), most averaging about 3 terabytes, but one would have in excess of 20 terabytes. We are looking at using 320 to 500 gigabyte drives in these arrays. (How? IEEE-1394. Which brings a question I will ask in a second email.)
Is RAID 5 all that bad using this software method? Is RAID 5 available?
Trever Adams
--
"They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -- Benjamin Franklin, 1759Quoted 5 lines
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/Quoted 5 lines
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/--
"Assassination is the extreme form of censorship." -- George Bernard Shaw (1856-1950)
Signature · 2 lines
<a href="http://www.edusupport.nl">EduSupport: Linux Desktop for schools and small to medium business in The Netherlands and Belgium</a>
Quoted 4 lines · Norbert van Nobelen
RAID5 in software works pretty good (survived a failed disk, and recovered another failing raid in 1 month). Hardware is better since you don't have a boot partition left which is usually just present on one disk (you can mirror that yourself ofcourse).
Err, you _can_ boot completely from a software RAID, it just takes a bit more work. I have an old PowerMac G4 400MHz with a Promise 20268 controller and 3 80GB drives booting from a software RAID. You just set up a 250-500MB boot partition mirrored with RAID 1 across all drives, then set up a RAID 5 swap partition and a RAID 5 LVM partition on each drive. Once LVM is configured with each remaining filesystem, install your distro (The new Debian-installer does very well) and set up Yaboot/GRUB/whatever to install a boot sector on each drive. Then set up a RAID+LVM initrd (Debian does this mostly automatically too), and reboot. This computer boots a custom 2.6.8.1 kernel, has 896MB RAM, and a 400MHz CPU, but it reads 41.5MiByte/sec from its RAID 5 partition with a 1MiByte blocksize, and has 16.8MiByte/sec over LVM over RAID 5 with the same blocksize. I've been following the discussions on 2.6 instability and "New development model" problems, but AFAICT, 2.6 has been rock stable on this box, which acts as an IPv4/IPv6 router/firewall/server.
Cheers,
Kyle Moffett
-----BEGIN GEEK CODE BLOCK-----
Version: 3.12
GCM/CS/IT/U d- s++: a18 C++++>$ UB/L/X/*++++(+)>$ P+++(++++)>$ L++++(+++) E W++(+) N+++(++) o? K? w--- O? M++ V? PS+() PE+(-) Y+ PGP+++ t+(+++) 5 X R? tv-(--) b++++(++) DI+ D+ G e->++++$ h!*()>++$ r !y?(-) ------END GEEK CODE BLOCK------
md and RAID 5 [was Re: LVM2]
In reply to Trever L. AdamsThank you all for having been so kind in your responses and help.
However, there is one more set of questions I have.
Does the md (software raid) have disk size or raid volume limits?
If I am using such things as USB or 1394 disks, is there a way to use labels in /etc/raidtab and with the tools so that when the disks, if they do, get renumbered in /dev that all works fine. I am aware that the kernel will autodetect these devices, but that the raidtab needs to be consistent. This is what I am trying to figure out how to do.
Thank you,
Trever Adams
--
"A modest woman, dressed out in all her finery, is the most tremendous object in the whole creation." -- Goldsmith
Re: md and RAID 5 [was Re: LVM2]
Quoted 11 lines · Trever L. Adams
Thank you all for having been so kind in your responses and help.
However, there is one more set of questions I have.
Does the md (software raid) have disk size or raid volume limits?
If I am using such things as USB or 1394 disks, is there a way to use labels in
/etc/raidtaband with the tools so that when the disks, if they do, get renumbered in /dev that all works fine. I am aware that the kernel will autodetect these devices, but that the raidtab needs to be consistent. This is what I am trying to figure out how to do.
EVMS can handle this for you. I've used it with a raid set I made with some of the drives being on usb and some on ide.
If you use evms, you might want to consider the activation after all disks have been discovered.
Signature · 1 line
Lab tests show that use of micro$oft causes cancer in lab animals
Re: md and RAID 5 [was Re: LVM2]
In reply to Trever L. AdamsQuoted 5 lines · On Friday January 21, [email protected] wrote:
Thank you all for having been so kind in your responses and help.
However, there is one more set of questions I have.
Does the md (software raid) have disk size or raid volume limits?
2^31 sectors for individual disks. Arrays do not have this limit.
Quoted 6 lines
If I am using such things as USB or 1394 disks, is there a way to use labels in
/etc/raidtaband with the tools so that when the disks, if they do, get renumbered in /dev that all works fine. I am aware that the kernel will autodetect these devices, but that the raidtab needs to be consistent. This is what I am trying to figure out how to do.
Scrap raidtools and /etc/raidtab. Explore "mdadm" and /etc/mdadm.conf
NeilBrown