Max storage size per system
Hello,
what is the absolut maximum storage capacity that can be managed on a single Linux 2.4 system with LVM?
Kind regards,
Stephan
Quoted 5 lines · Stephan Austermuehle
Hello,
what is the absolut maximum storage capacity that can be managed on a single Linux 2.4 system with LVM?
man vgcreate
-s, --physicalextentsize PhysicalExtentSize[kKmMgGtT]
Sets the physical extent size on physical volumes of this
volume group.
A size suffix (k for kilobytes up to t for terabytes)
is optional,
megabytes is the default if no suffix is present. Values
can be from 8
KB to 16 GB in powers of 2. The default of 4 MB causes
maximum LV sizes
of ~256GB because as many as ~64k extents are supported
per LV. In case
larger maximum LV sizes are needed (later), you need to
set the PE size
to a larger value as well. Later changes of the PE size
in an existing
VG are not supported.
So, this means you can allocate
64k * 16G == 64*1024 * 16*1024*1024*1024 == 1125899906842624 bytes == 1024Tera Bytes,
Hovever,
To limit kernel memory usage, there is a limit of 65536 physical
extents (PE)
per logical volume, so the PE size determines the maximum
logical volume size.
The default PE size of 4MB limits a single logical volume to
256GB (see the -s
option to raise that limit). There is also (as of Linux 2.4) a
kernel limita
tion of 2TB per block device.
So, this would mean 2TB pr. LV, but there can also be a number of LV's. I dont remember how many, but proberly a power of 2, like 256 or 64k. But wait, there is more...
You can have more than one VG system on your linux (no idea how many, i have had 2)
So, all in all, i would asume you can have MANY TERABYTES. Possibly Penta? bytes ?
How much space do you need anyway ?
JonB
Quoted 1 line · Jon Bendtsen
So, this would mean 2TB pr. LV, but there can also be a number of LV's.
Okay, so the LVM can manage really large disk storage. Does anybody know whether the number of mounted filesystem per Linux system is limited?
Quoted 1 line
How much space do you need anyway ?
100+ Terabytes
Kind regards,
Stephan
Quoted 10 lines · Jon Bendtsen
Quoted 2 lines
So, this would mean 2TB pr. LV, but there can also be a number of LV's.
Okay, so the LVM can manage really large disk storage. Does anybody know whether the number of mounted filesystem per Linux system is limited?
Quoted 1 line
How much space do you need anyway ?
100+ Terabytes
Hm, sounds like lots. :) What kind of devices will you be running with. Even with 300GB drives you will need 300-400 of them to get
100TB.
Quoted 3 lines
Kind regards,
Stephan
--------
PGP public key: https://tnonline.net/secure/pgp_key.txt
Quoted 3 lines · Anders Widman
Hm, sounds like lots. :) What kind of devices will you be running with. Even with 300GB drives you will need 300-400 of them to get100TB.
Easily achieved w/ such technologies as iscsi and hyperscsi (;
Signature · 3 lines
Jesse Keating RHCE MCSE Pogo Linux -- Support Tech tel: (888) 828-7646 ex: 436
Quoted 1 line · Jesse Keating
Easily achieved w/ such technologies as iscsi and hyperscsi (;
Which are nothing but trash.
Kind regards,
Stephan
Quoted 3 lines · Stephan Austermuehle
Quoted 1 line · Jesse Keating
Easily achieved w/ such technologies as iscsi and hyperscsi (;
Which are nothing but trash.
Hey man, don't pull any punches, tell us how you *Really* feel.
Signature · 4 lines
"On two occasions, I have been asked [by members of Parliament], 'Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?' I am not able to rightly apprehend the kind of confusion of ideas that could provoke such a question." -- Charles Babbage
Quoted 1 line · Petro
Hey man, don't pull any punches, tell us how you *Really* feel.
Well, iSCSI was developed and pushed by some networking guys with Ethernet in mind to get into the storage market without developing know how for a really efficient data link. So iSCSI has to handle lots of
overhead: SCSI commands are encapsulated in IP, IP is encapuslated in Ethernet. This leads to bad performance (latency, throughput) compared to a Fibre Channel connection with the same link speed. Maybe the performance impact will become negligible with 10G or 100G Ethernet but at the moment we don't have these technologies available.
Stephan
On Friday 07 March 2003 06:20, Stephan Austermuehle uttered:
Quoted 8 lines
Well, iSCSI was developed and pushed by some networking guys with Ethernet in mind to get into the storage market without developing know how for a really efficient data link. So iSCSI has to handle lots of
overhead: SCSI commands are encapsulated in IP, IP is encapuslated in Ethernet. This leads to bad performance (latency, throughput) compared to a Fibre Channel connection with the same link speed. Maybe the performance impact will become negligible with 10G or 100G Ethernet but at the moment we don't have these technologies available.
This speaks nothing of HyperSCSI which is indepenant of TCP/IP and all of it's ugly overhead. Hyperscsi runs in raw ethernet, no TCP/IP even need be on the wire. Data writing w/ HyperSCSI is far faster than with NFS or iSCSI.
Signature · 3 lines
Jesse Keating RHCE MCSE Pogo Linux -- Support Tech tel: 888.828.7646
Quoted 3 lines · Anders Widman
Hm, sounds like lots. :) What kind of devices will you be running with. Even with 300GB drives you will need 300-400 of them to get100TB.
A single HP XP 1024 (which is an OEM product of a HDS Lightning) stores 130 TB with a 7p1 RAID5 configuration and 146 GB disks.
Kind regards,
Stephan
Quoted 8 lines · Stephan Austermuehle
Quoted 3 lines · Anders Widman
Hm, sounds like lots. :) What kind of devices will you be running with. Even with 300GB drives you will need 300-400 of them to get100TB.
A single HP XP 1024 (which is an OEM product of a HDS Lightning) stores 130 TB with a 7p1 RAID5 configuration and 146 GB disks.
hmm, that would need 130T/0.146T = 890 disks. And running raid 5... personaly i dont like raid5, because if you loose more than one disk your data are gone, and with this ammount of disks, i expect failure rather often. (it's proberly not one big raid5 array, but that just makes it need more disks).
JonB
Quoted 1 line · On Thu, Mar 06, 2003 at 05:21:50PM +0100, [email protected] wrote:
hmm, that would need 130T/0.146T = 890 disks. And running raid 5...
Please have a look at the following links. A single HDS Lightning (aka HP XP 1024) can address up to 1,024 disks.
http://www.hp.com/products1/storage/products/disk_arrays/highend/xp1024/index.html http://www.hds.com/products/systems/9900v/
Quoted 4 lines
personaly i dont like raid5, because if you loose more than one disk your data are gone, and with this ammount of disks, i expect failure rather often. (it's proberly not one big raid5 array, but that just makes it need more disks).
I prefer RAID 1 (or RAID 10), too, but sometimes one has to use RAID 5. A common strategy is to put a higher number of hot spare drives in one system. Another (somewhat newer) strategy is to use two rotating parity drives per RAID set.
Stephan
Quoted 11 lines · Stephan Austermuehle
Quoted 1 line · On Thu, Mar 06, 2003 at 05:21:50PM +0100, [email protected] wrote:
hmm, that would need 130T/0.146T = 890 disks. And running raid 5...
Please have a look at the following links. A single HDS Lightning (aka HP XP 1024) can address up to 1,024 disks.
http://www.hp.com/products1/storage/products/disk_arrays/highend/xp1024/index.html http://www.hds.com/products/systems/9900v/
nice, imagien all that porn ;-D
Quoted 9 lines
Quoted 4 lines
personaly i dont like raid5, because if you loose more than one disk your data are gone, and with this ammount of disks, i expect failure rather often. (it's proberly not one big raid5 array, but that just makes it need more disks).
I prefer RAID 1 (or RAID 10), too, but sometimes one has to use RAID 5. A common strategy is to put a higher number of hot spare drives in one system. Another (somewhat newer) strategy is to use two rotating parity drives per RAID set.
2 rotating parity disks ? Can you elaborate on how that works ? I have thought about using something else than raid5, where you would split the data across a number of disks, AND use a mirrored device for storing the parity on.
JonB
Quoted 1 line · Jon Bendtsen
2 rotating parity disks ? Can you elaborate on how that works ?
"The Mathematics behind the RAID 5DP":
http://www.hp.com/products1/storage/products/disk_arrays/infolibrary/mathematics_behind_RAID_5DP.pdf
"Analysis of RAID 5DP":
http://www.hp.com/products1/storage/products/disk_arrays/infolibrary/analysis_0f_RAID_5DP.pdf
Kind regards,
Stephan
Quoted 5 lines · Stephan Austermuehle
Quoted 1 line · Jon Bendtsen
2 rotating parity disks ? Can you elaborate on how that works ?
Quoted 7 lines
"The Mathematics behind the RAID 5DP":
http://www.hp.com/products1/storage/products/disk_arrays/infolibrary/mathematics_behind_RAID_5DP.pdf
"Analysis of RAID 5DP":
http://www.hp.com/products1/storage/products/disk_arrays/infolibrary/analysis_0f_RAID_5DP.pdf
thanks.
JonB
Hello,
Quoted 2 lines · Stephan Austermuehle
Okay, so the LVM can manage really large disk storage. Does anybody know whether the number of mounted filesystem per Linux system is limited?
yes, it is limited through the kernel. There are two ways out of it:
a) increase the number (fiddle through the source, recompile, reboot)
b) use the automounter
If you don't need all filesystems mounted at once I'd suggest the second approach. (e.g. for samba or backup store, whatever)
Signature · 3 lines
Regards, Wiktor Wodecki
[linux-lvm] can i extend a volumen bigger 256 gigas ?thx
Hello,
It is possible to extend a volumen more than 256 gigas? it doesnt let me grow the space.
this is muy info :
root]# cat /proc/lvm/global
LVM module LVM version 1.0.3(19/02/2002)
Total: 1 VG 2 PVs 1 LV (1 LV open 1 times)
Global: 449424 bytes malloced IOP version: 10 1175 days 12:40:03 active
VG: gl [2 PV, 1 LV/1 open] PE Size: 4096 KB
Usage [KB/PE]: 294883328 /71993 total 113950720 /27820 used 180932608
/44173 free
PVs: [AA] ide/host0/bus0/target0/lun0/part4 113950720 /27820 113950720
/27820 0 /0
[AA] ide/host0/bus0/target1/lun0/part1 180932608 /44173 0 /0
180932608 /44173
LV: [AWDL ] vol1 113950720 /27820 1x openi have installed: lvm-1.0.1-2mdk.i586.rpm
thanks on advanced.
Re: [linux-lvm] can i extend a volumen bigger 256 gigas ?thx
when you create your volume group you can change the extent size with the -s switch. recent versions of LVM use 32MB as the default which gives you a max LV of about 2TB. the old default extent size of 4MB gave you about 255GB max LV size.
Quoted 31 lines · fnt
Hello,
It is possible to extend a volumen more than 256 gigas? it doesnt let me grow the space.
this is muy info :
root]# cat/proc/lvm/global
LVM module LVM version 1.0.3(19/02/2002)Total: 1 VG 2 PVs 1 LV (1 LV open 1 times)
Global: 449424 bytes malloced IOP version: 10 1175 days 12:40:03 active
VG: gl [2 PV, 1 LV/1 open] PE Size: 4096 KB
Usage [KB/PE]: 294883328 /71993 total 113950720 /27820 used 180932608
/44173 free
PVs: [AA] ide/host0/bus0/target0/lun0/part4 113950720 /27820 113950720
/27820 0 /0[AA] ide/host0/bus0/target1/lun0/part1 180932608 /44173 0 /0 180932608 /44173 LV: [AWDL ] vol1 113950720 /27820 1x openi have installed: lvm-1.0.1-2mdk.i586.rpm
thanks on advanced.
_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Signature · 10 lines
==================================== Erik Nugent System Administrator Department of Computer Science The University of Montana, Missoula (406) 243-2812 ==================================== "Tee off and fly freely" "Steady" Ed ====================================
Re: [linux-lvm] can i extend a volumen bigger 256 gigas ?thx
In reply to fntYes, it is possible. What's the command you're using and the error you get? The maximum size of a logical volume is a function of the volume group physical extent size (I don't remember the details). You may need to create the volume group with a larger physical extent in order to have larger capacity logical volumes, for instance, 'vgcreate -s 16m ...'. As far as I know, you do have to recreate the volume group in order to change the physical extent size. Hope this helps.
---
Daniel Barbar
Quoted 31 lines · fnt
Hello,
It is possible to extend a volumen more than 256 gigas? it doesnt let me grow the space.
this is muy info :
root]# cat/proc/lvm/global
LVM module LVM version 1.0.3(19/02/2002)Total: 1 VG 2 PVs 1 LV (1 LV open 1 times)
Global: 449424 bytes malloced IOP version: 10 1175 days 12:40:03 active
VG: gl [2 PV, 1 LV/1 open] PE Size: 4096 KB
Usage [KB/PE]: 294883328 /71993 total 113950720 /27820 used 180932608
/44173 free
PVs: [AA] ide/host0/bus0/target0/lun0/part4 113950720 /27820 113950720
/27820 0 /0[AA] ide/host0/bus0/target1/lun0/part1 180932608 /44173 0 /0 180932608 /44173 LV: [AWDL ] vol1 113950720 /27820 1x openi have installed: lvm-1.0.1-2mdk.i586.rpm
thanks on advanced.
_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Signature · 1 line
Daniel Barbar
Re: [linux-lvm] can i extend a volumen bigger 256 gigas ?thx-thanks for ur time
thx for ur time.
----- Original Message -----
From: "Daniel Barbar" <[email protected]>
To: <[email protected]>
Sent: Monday, May 05, 2003 10:40 PM
Subject: Re: [linux-lvm] can i extend a volumen bigger 256 gigas ?thx
Quoted 9 lines
Yes, it is possible. What's the command you're using and the error you get? The maximum size of a logical volume is a function of the volume group physical extent size (I don't remember the details). You may need to create the volume group with a larger physical extent in order to have larger capacity logical volumes, for instance, 'vgcreate -s 16m ...'. As far as I know, you do have to recreate the volume group in order to change the physical extent size. Hope this helps.
Quoted 15 lines
---
Daniel BarbarQuoted 11 lines · fnt
Hello,
It is possible to extend a volumen more than 256 gigas? it doesnt let me grow the space.
this is muy info :
root]# cat/proc/lvm/global
LVM module LVM version 1.0.3(19/02/2002)Total: 1 VG 2 PVs 1 LV (1 LV open 1 times)
Global: 449424 bytes malloced IOP version: 10 1175 days 12:40:03
active
Quoted 3 lines
Quoted 3 lines
VG: gl [2 PV, 1 LV/1 open] PE Size: 4096 KB
Usage [KB/PE]: 294883328 /71993 total 113950720 /27820 used
180932608
Quoted 2 lines
Quoted 2 lines
/44173 free
PVs: [AA] ide/host0/bus0/target0/lun0/part4 113950720 /27820
113950720
Quoted 2 lines
Quoted 2 lines
/27820 0 /0
[AA] ide/host0/bus0/target1/lun0/part1 180932608 /44173
0 /0
Quoted 23 lines
Quoted 5 lines
180932608 /44173
LV: [AWDL ] vol1 113950720 /27820 1x openi have installed: lvm-1.0.1-2mdk.i586.rpm
Quoted 8 lines
thanks on advanced.
_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/--
Daniel Barbar_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] can i extend a volumen bigger 256 gigas ?thx
In reply to Daniel BarbarHi,
Recreating the VG , i will keep the data?
thx
Joaquin Ign.
Best Regads-
---- Original Message -----
From: "Daniel Barbar" <[email protected]>
To: <[email protected]>
Sent: Monday, May 05, 2003 10:40 PM
Subject: Re: [linux-lvm] can i extend a volumen bigger 256 gigas ?thx
Quoted 9 lines
Yes, it is possible. What's the command you're using and the error you get? The maximum size of a logical volume is a function of the volume group physical extent size (I don't remember the details). You may need to create the volume group with a larger physical extent in order to have larger capacity logical volumes, for instance, 'vgcreate -s 16m ...'. As far as I know, you do have to recreate the volume group in order to change the physical extent size. Hope this helps.
Quoted 15 lines
---
Daniel BarbarQuoted 11 lines · fnt
Hello,
It is possible to extend a volumen more than 256 gigas? it doesnt let me grow the space.
this is muy info :
root]# cat/proc/lvm/global
LVM module LVM version 1.0.3(19/02/2002)Total: 1 VG 2 PVs 1 LV (1 LV open 1 times)
Global: 449424 bytes malloced IOP version: 10 1175 days 12:40:03
active
Quoted 3 lines
Quoted 3 lines
VG: gl [2 PV, 1 LV/1 open] PE Size: 4096 KB
Usage [KB/PE]: 294883328 /71993 total 113950720 /27820 used
180932608
Quoted 2 lines
Quoted 2 lines
/44173 free
PVs: [AA] ide/host0/bus0/target0/lun0/part4 113950720 /27820
113950720
Quoted 2 lines
Quoted 2 lines
/27820 0 /0
[AA] ide/host0/bus0/target1/lun0/part1 180932608 /44173
0 /0
Quoted 23 lines
Quoted 5 lines
180932608 /44173
LV: [AWDL ] vol1 113950720 /27820 1x openi have installed: lvm-1.0.1-2mdk.i586.rpm
Quoted 8 lines
thanks on advanced.
_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/--
Daniel Barbar_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
Re: [linux-lvm] can i extend a volumen bigger 256 gigas ?thx
In reply to fntQuoted 2 lines · fnt
It is possible to extend a volumen more than 256 gigas?
Yes.
Quoted 6 lines
it doesnt let me grow the space.
this is muy info :
root]# cat/proc/lvm/global
VG: gl [2 PV, 1 LV/1 open] PE Size: 4096 KB
Usage [KB/PE]: 294883328 /71993 total 113950720 /27820 used 180932608
Directly from vgcreate(8) (lvm2 tools, version 1.95.15-1 (Debian Sid)):
To limit kernel memory usage, there is a limit of 65536 physical extents (PE) per logical volume, so the PE size determines the maximum logical volume size. The default PE size of 4MB limits a single logi- cal volume to 256GB (see the -s option to raise that limit). There is also (as of Linux 2.4) a kernel limitation of 2TB per block device.
As far as I remember, this limit is also there in LVM1.
Hope it helps.
Signature · 2 lines
Jose Luis Domingo Lopez Linux Registered User #189436 Debian Linux Sid (Linux 2.5.68)
Quoted 6 lines · Stephan Austermuehle
Quoted 1 line · Jon Bendtsen
So, this would mean 2TB pr. LV, but there can also be a number of LV's.
Okay, so the LVM can manage really large disk storage. Does anybody know whether the number of mounted filesystem per Linux system is limited?
i think you need to read up on how LVM works. In the bottom you have pv's. physical volumes. They are divided into physical extends which can be from 4k to 16G. But you set this for the hole VolumenGroup. The VG has a name, and can span several physical devices. Inside this VG, you make a LV, LogicalVolumen. The LV can hold 64k PE, physical extends, which if you make them 16G... means 1024T ? however, the LV is a blockdevice, and thus an LV cant be bigger than 2T. So you would have to use many LV's to store 100+T.
Quoted 3 lines
Quoted 1 line
How much space do you need anyway ?
100+ Terabytes
What do you need it for ?
JonB
Maximum of 256 LVs * 2TB in stock 2.4 = 512TB.
Quoted 8 lines · Stephan Austermuehle
Hello,
what is the absolut maximum storage capacity that can be managed on a single Linux 2.4 system with LVM?
Kind regards,
Stephan
Signature · 15 lines
Regards,
Heinz -- The LVM Guy --
*** Software bugs are stupid.
Nevertheless it needs not so stupid people to solve them ***
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Heinz Mauelshagen Sistina Software Inc.
Senior Consultant/Developer Am Sonnenhang 11
56242 Marienrachdorf
Germany
[email protected] +49 2626 141200
FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-