*** ANNOUNCEMENT *** LVM 1.0.3 available at www.sistina.com
*** ANNOUNCEMENT *** LVM 1.0.3 available at www.sistina.com
Hi all,
LVM 1.0.3 supports both version 1 and 2 of the metadata.
There's *no* need to run any metadata update tools.
A tarball is available now at
for download (Follow the "LVM 1.0" link).
Changes to LVM 1.0.2 include:
o vgcfgrestore supports restores to different sized physical volumes; useful in cases where a replacement disk is a little bit to large or for test purposes; option enhancements; physical volume UUID restore fix
o vgchange removes failed snapshots and activates the volume group rather than failing on it. Optionally forces device number changes in cases where it finds clashes so that the volume group can be activated
o vgexport exports volume groups not showing up in /etc/lvmtab*
o vgscan can drop all snapshots or those in a particular volume group now
o "pvmove -i" ignores read errors while moving and supports moves in inactive volume groups
o > 1 TB fixes for physical and logical volumes (2 TB limitation on 2.4 persists)
o more...
See the CHANGELOG file contained in the tarball for further information.
Feed back LVM related information to <[email protected]>.
Thanks a lot for your support of LVM.
Awesome! The new vgchange behavior is precisely what I needed to save my ass from the apocalyptic corruption problems between bad-VM and snapshots.. very happy to see this in release.
Quoth Heinz J . Mauelshagen:
Quoted 51 lines
*** ANNOUNCEMENT *** LVM 1.0.3 available at www.sistina.com
Hi all,
LVM 1.0.3 supports both version 1 and 2 of the metadata.
There's *no* need to run any metadata update tools.
A tarball is available now at
<http://www.sistina.com/>for download (Follow the "LVM 1.0" link).
Changes to LVM 1.0.2 include:
o vgcfgrestore supports restores to different sized physical volumes; useful in cases where a replacement disk is a little bit to large or for test purposes; option enhancements; physical volume UUID restore fix
o vgchange removes failed snapshots and activates the volume group rather than failing on it. Optionally forces device number changes in cases where it finds clashes so that the volume group can be activated
o vgexport exports volume groups not showing up in
/etc/lvmtab*o vgscan can drop all snapshots or those in a particular volume group now
o "pvmove -i" ignores read errors while moving and supports moves in inactive volume groups
o > 1 TB fixes for physical and logical volumes (2 TB limitation on 2.4 persists)
o more...
See the CHANGELOG file contained in the tarball for further information.
Feed back LVM related information to <[email protected]>.
Thanks a lot for your support of LVM.
_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html
Signature · 2 lines
Don't hate me because I'm beautiful.
Hate me because I run your IT department.Quoted 4 lines · On Tue, Feb 19, 2002 at 12:14:53PM -0800, [email protected] wrote:
Awesome! The new vgchange behavior is precisely what I needed to save my ass from the apocalyptic corruption problems between bad-VM and snapshots.. very happy to see this in release.
Happy to hear that it helps someone in these hard VM days ;-)
Kind reagrds,
Heinz -- The LVM Guy --
Quoted 46 lines
Quoth Heinz J . Mauelshagen:
Quoted 43 lines
*** ANNOUNCEMENT *** LVM 1.0.3 available at www.sistina.com
Hi all,
LVM 1.0.3 supports both version 1 and 2 of the metadata.
There's *no* need to run any metadata update tools.
A tarball is available now at
<http://www.sistina.com/>for download (Follow the "LVM 1.0" link).
Changes to LVM 1.0.2 include:
o vgcfgrestore supports restores to different sized physical volumes; useful in cases where a replacement disk is a little bit to large or for test purposes; option enhancements; physical volume UUID restore fix
o vgchange removes failed snapshots and activates the volume group rather than failing on it. Optionally forces device number changes in cases where it finds clashes so that the volume group can be activated
o vgexport exports volume groups not showing up in
/etc/lvmtab*o vgscan can drop all snapshots or those in a particular volume group now
o "pvmove -i" ignores read errors while moving and supports moves in inactive volume groups
o > 1 TB fixes for physical and logical volumes (2 TB limitation on 2.4 persists)
o more...
See the CHANGELOG file contained in the tarball for further information.
Feed back LVM related information to <[email protected]>.
Quoted 18 lines
Quoted 8 lines
Thanks a lot for your support of LVM.
_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.htmlSignature · 8 lines
Don't hate me because I'm beautiful. Hate me because I run your IT department. _______________________________________________ linux-lvm mailing list [email protected] http://lists.sistina.com/mailman/listinfo/linux-lvm read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html
*** Software bugs are stupid.
Nevertheless it needs not so stupid people to solve them ***
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Heinz Mauelshagen Sistina Software Inc.
Senior Consultant/Developer Am Sonnenhang 11
56242 Marienrachdorf Germany
[email protected] +49 2626 141200
FAX 924446
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
[linux-lvm] lvm and 'poor mans raid' on heterogenous hard drives!
In reply to Heinz J . MauelshagenHmmmm,
Ok so I have 4 hard drives of very varying capacity (20G,12G,5G,4G)
I have an off-board IDE controller (Promise 66)
I think 'maybe I can take advantage of the IDE system and put a drive on each master and stripe them.' A hardware RAID card would be suboptimal because (as I understand it) the striped volume could only be as big as the smallest drive.
So, I set up the drives as;
hda = 20G
hda1 = 64M and is /boot hda2 = 1G hdc = 12G hdc1 = 1G hde = 5G hde1 = 1G hdg = 4G hdg1 = 1G
The other partitions are set up for various other volume groups and non LVM system partitions.
hda2, hdc1, hde1, hdg1 are added to a VG 'fast'
The other LVM partitions on hdc-g are in a volume group that doesn't see much traffic, just storage.
I make logical volumes on fast striped across 4 drives. So thats 4 partitions at the beginning of the drives, with volumes striped across them.
I expected some performance improvements.
So I get the iozone benchmark and start running it on the system. I compare performance of striped volumes with performance of a volume purely on hda.
Interestingly, the performance improvement is not dramatic, I expected better. The main improvement seems to be that the striped volumes do better with larger file sizes and as file size increases the striped volume keeps its performance up better.
I'm a newbie at this benchmarking lark, so if anyone wants the excel spreadsheets generated from iozone just ask, I'd like a second opinion! There are some strange things, like when file size gets above 16M performance drops dramatically regardless of striped or linear volumes... Maybe its something to do with extents?
Re: [linux-lvm] lvm and 'poor mans raid' on heterogenous hard drives!
Quoted 6 lines · Steve Wray
Ok so I have 4 hard drives of very varying capacity I think 'maybe I can take advantage of the IDE system and put a drive on each master and stripe them.'
I expected some performance improvements. Interestingly, the performance improvement is not dramatic, I expected better.
Well, basically you cannot get performace much better than 4x the slowest disk. The reason would be that because it is striped across all 4 disks, no matter how fast the other disks run you will always have to end up waiting for the slowest disk to complete every 4th block I/O.
I think you will find that the performance of modern UDMA disks is far better than any of the older disks, so you are better off just using the single large disk for best performance & reliability.
Quoted 3 lines
The main improvement seems to be that the striped volumes do better with larger file sizes and as file size increases the striped volume keeps its performance up better.
The other problem (as I always complain about whenever people try striping and it doesn't work) is that unless you do large file I/O (as you have seen) you don't get much performance gains. This is because for each small* read you basically have to wait for the maxiumu seek time of all of the disks to do a read. For normal I/O patterns this is really bad.
(*) small means larger than the stripe size and less than maybe 8-12
stripes.
You also have the problem that you are 4x as likely to lose all of your data in this case.
Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/
RE: [linux-lvm] lvm and 'poor mans raid' on heterogenous hard drives!
Quoted 7 lines
From: Andreas Dilger [mailto:[email protected]]
Quoted 4 lines · Steve Wray
Ok so I have 4 hard drives of very varying capacity I think 'maybe I can take advantage of the IDE system and put a drive on each master and stripe them.'
Quoted 8 lines
Quoted 2 lines
I expected some performance improvements. Interestingly, the performance improvement is not dramatic, I expected better.
Well, basically you cannot get performace much better than 4x the slowest disk. The reason would be that because it is striped across all 4 disks, no matter how fast the other disks run you will always have to end up waiting for the slowest disk to complete every 4th block I/O.
Yes, I think I've factored that in; While the disks are very heterogenous, the motherboard is an old VIA chipset, from TMI. Its onboard controller is UDMA33. The 2 drives that are plugged into it are both capable of UDMA66. Sometime I'll get a newer muthaboard :)
Meanwhile, the promise card is at UDMA66, only one of the drives plugged into it is capable of 66 and its only on 40 cable, so its working at 33 as well. The other drive only goes at 33 too.
So, overall, the system is UDMA33 which I figured would be minimising the impact of a heterogenous architecture.
But obviously all this DOES introduce piles of other variables! Differing cache in the drives for one thing.
Quoted 3 lines
I think you will find that the performance of modern UDMA disks is far better than any of the older disks, so you are better off just using the single large disk for best performance & reliability.
yeah but the m/board won't help the new drive. I think that the 20G drive is quite a bit newer than the MB (which I think was one of the first ATX boards out. The bios even crashes at the chipset screen 8) Yes, and I'm trying to benchmark this mongrel. Ah well I'm learning!
Quoted 10 lines
Quoted 3 lines
The main improvement seems to be that the striped volumes do better with larger file sizes and as file size increases the striped volume keeps its performance up better.
The other problem (as I always complain about whenever people try striping and it doesn't work) is that unless you do large file I/O (as you have seen) you don't get much performance gains. This is because for each small* read you basically have to wait for the maxiumu seek time of all of the disks to do a read. For normal I/O patterns this is really bad.
This is very very true. I'm having second thoughts about having all of /var on it. Maybe seperate some of the /var directories into their own striped volumes.
But what do you think of the huge drop in performance at file sizes of 16M and up (at all block sizes)?
It goes from 50Mps down to less than 20Mps starting when the file size hits 16M? Looking at the figures, it virtually halves.
Read is even more dramatic from 108213Kps at 8192K files down to 14796Kps at 16384K files!
Now this is the same, striped or non striped; striping just smoothes things out on either side of the step!
All the filesystems are LVM at standard extent size with ext3 filesystems and default journalling.
Uh oh, this is getting off topic! (LVM)
Quoted 5 lines
(*) small means larger than the stripe size and less than maybe 8-12
stripes.You also have the problem that you are 4x as likely to lose all of your data in this case.
yeah but its only /var, /usr/lib, /usr/share, /tmp, swap that sort of thing. I think swap may have been a mistake looking at the benchmark!
Re: [linux-lvm] lvm and 'poor mans raid' on heterogenous hard drives!
Quoted 10 lines · Steve Wray
Quoted 6 lines
The other problem (as I always complain about whenever people try striping and it doesn't work) is that unless you do large file I/O (as you have seen) you don't get much performance gains. This is because for each small* read you basically have to wait for the maxiumu seek time of all of the disks to do a read. For normal I/O patterns this is really bad.
This is very very true. I'm having second thoughts about having all of /var on it. Maybe seperate some of the /var directories into their own striped volumes.
In most applications, you are better off to put separate trees each on their own drive. Usually you only have a single application writing into each tree (e.g. sendmail writing to /var/spool/{mail,mqueue}, other programs writing to /var/tmp, lpd writing to /var/spool/lpd, etc). If you have each of the high-volume trees on a separate drive it means that each app can write at the full disk bandwidth without much seeking, instead of the striped case where each app needs to seek every drive for every write.
Quoted 6 lines
But what do you think of the huge drop in performance at file sizes of 16M and up (at all block sizes)?
It goes from 50Mps down to less than 20Mps starting when the file size hits 16M? Looking at the figures, it virtually halves.
Read is even more dramatic from 108213Kps at 8192K files down to 14796Kps at 16384K files!
Could be several things - cache size issues, journal size, maybe once you are reading large enough files and your bus/CPU/cache can't keep up you need to skip a full disk revolution for each subsequent read...
Quoted 5 lines
Quoted 2 lines
You also have the problem that you are 4x as likely to lose all of your data in this case.
yeah but its only /var,
/usr/lib,/usr/share, /tmp, swap that sort of thing. I think swap may have been a mistake looking at the benchmark!
Swap is a bad move, since you can just add multiple swap spaces with the same priority (if you so choose) and it will do the striping for you. Likewise, you could put each of the above trees on their own drive and you would probably get better overall performance than striping.
Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/
RE: [linux-lvm] lvm and 'poor mans raid' on heterogenous hard drives!
Quoted 15 lines
From: [email protected] [mailto:[email protected]]On Behalf Of Andreas Dilger
Quoted 10 lines · Steve Wray
Quoted 6 lines
The other problem (as I always complain about whenever people try striping and it doesn't work) is that unless you do large file I/O (as you have seen) you don't get much performance gains. This is because for each small* read you basically have to wait for the maxiumu seek time of all of the disks to do a read. For normal I/O patterns this is really bad.
This is very very true. I'm having second thoughts about having all of /var on it. Maybe seperate some of the /var directories into their own striped volumes.
Quoted 8 lines
In most applications, you are better off to put separate trees each on their own drive. Usually you only have a single application writing into each tree (e.g. sendmail writing to
/var/spool/{mail,mqueue}, other programs writing to/var/tmp, lpd writing to/var/spool/lpd, etc). If you have each of the high-volume trees on a separate drive it means that each app can write at the full disk bandwidth without much seeking, instead of the striped case where each app needs to seek every drive for every write.
Ohhhhh. Now thats a consideration I didn't take into account. Thanks for the insight. Next time I reinstall this sucker, I'll give it a go. Plus the parallel swap suggestion (which I also just read in the software RAID howto). Pity I can't just resize the partitions and insert somenew ones! (I doubt that partition magic will successfully move'n'resize an LVM partition! (even if I did have a windoze install on that box)).
Quoted 10 lines
Quoted 6 lines
But what do you think of the huge drop in performance at file sizes of 16M and up (at all block sizes)?
It goes from 50Mps down to less than 20Mps starting when the file size hits 16M? Looking at the figures, it virtually halves.
Read is even more dramatic from 108213Kps at 8192K files down to 14796Kps at 16384K files!Could be several things - cache size issues, journal size, maybe once you are reading large enough files and your bus/CPU/cache can't keep up you need to skip a full disk revolution for each subsequent read...
It seems dependent on system memory. I increased it from 68M to 192M and ran the same benchmarks; the 8M-16M step was displaced to 32M-64M. The really interesting thing is that the step is stable across all block sizes (as near as I can tell). Ie; before the step, blocksize is very important (small block sizes are way faster) after the step it doesn't matter what the blocksize is, the step goes down to 20Mps.
I know this is unlikely to be LVM related tho. I can't see that this has anything to do with extent size or whatever, and its nothing to do with striping vs linear. So I guess I better shut up about it on this list...
:)
Quoted 25 lines
Quoted 4 lines
Quoted 2 lines
You also have the problem that you are 4x as likely to lose all of your data in this case.
yeah but its only /var,
/usr/lib,/usr/share, /tmp, swap thatsort of thing.
Quoted 1 line
I think swap may have been a mistake looking at the benchmark!
Swap is a bad move, since you can just add multiple swap spaces with the same priority (if you so choose) and it will do the striping for you. Likewise, you could put each of the above trees on their own drive and you would probably get better overall performance than striping.
Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html
[linux-lvm] striping volumes
Ok,
after a bit more benchmarking, it seems clear that striping logical volumes across multiple heterogenous IDE drives (all masters)
is only beneficial (WRT performance) on large file sizes. The file size at which the performance increase happens appears to depend on system memory. On my system at 128M RAM, the filesize is around 32M, with 64M RAM that drops to 8M files; but the point at which the step occurs is nothing to do with striping vs linear.
Also, there seems to be very very little difference
for *reads*; only writes (and the difference is definitely worth it. I get a 10Mps improvement over linear mapping starting at 64M files and that improvement is stable as filesize increases; linear mapping performance seems to constantly drop off as file size increases).
I guess that this makes sense all things considered.
What I'd like to know is;
has anyone else done any similar benchmarking on LVM? If so, where can I get a peek at it please! :)
Re: [linux-lvm] lvm and 'poor mans raid' on heterogenous hard drives!
In reply to Andreas Dilger-----BEGIN PGP SIGNED MESSAGE-----
Quoted 4 lines · Andreas Dilger
Swap is a bad move, since you can just add multiple swap spaces with the same priority (if you so choose) and it will do the striping for you. Likewise, you could put each of the above trees on their own drive and you would probably get better overall performance than striping.
This suggestion has always bothered me. Yes, if all you care about performance, setting up swap this way works fine. However, for every drive you add you increase the likelyhood that you're system will fail due to a drive failure (what happens when one of the swap slices suddenly dissapears?).
The better (IMO) suggestion is to use MD to setup RAID-1 mirrors and then swapadd those. That way, if one of your drives go south, you still have a workable swap.
- --
- -rupa
-----BEGIN PGP SIGNATURE-----
Version: 2.6.3ia
Charset: noconv
Comment: Processed by Mailcrypt 3.5.6, an Emacs/PGP interface
iQEVAwUBPHRCF3HDM4ucEopdAQHojAf/TbnHkeXcK6KwF+wQKOug8FcqJoNsZB8s 6XmCvL2V9QIdEMguXEob4qys/h2Km008xfXXvzIW+g8eXl/8zhoQAtQxRgBpJWZj Xa7XdzHUW2tEGTwztr5w/FJKFtL5nqqme6aYcILpbD2RFosgCSI8rm26QcgvSbXv /L6xmKx7ha5tD1QuMrh9/dVf5ei//c4BPiLeLSItmPaybm6DCL9NIYADQK+8WtLh WUDKycmjd+KTzFPiEH9Wmca+pbqE9XTEfmbsDRrTUSA9Fyeg3Y/t5KG4E1eI4HfS HtJvbtyYEY2uUAgiSPMcicbtxaOnyrJBoHGdBjmX6WK13lHQhoeQpA== =X3cg
-----END PGP SIGNATURE-----
Re: [linux-lvm] lvm and 'poor mans raid' on heterogenous hard drives!
Quoted 12 lines · Rupa Schomaker
Quoted 4 lines · Andreas Dilger
Swap is a bad move, since you can just add multiple swap spaces with the same priority (if you so choose) and it will do the striping for you. Likewise, you could put each of the above trees on their own drive and you would probably get better overall performance than striping.
This suggestion has always bothered me. Yes, if all you care about performance, setting up swap this way works fine. However, for every drive you add you increase the likelyhood that you're system will fail due to a drive failure (what happens when one of the swap slices suddenly dissapears?).
Quoted 3 lines
The better (IMO) suggestion is to use MD to setup RAID-1 mirrors and then swapadd those. That way, if one of your drives go south, you still have a workable swap.
Well that is true of ANY setup that uses RAID-0/stripe, or any unmirrored disk for that matter. I am by no means advocating the use of striped swap, just saying that to stripe at the physical layer is no benefit over letting the swap do its own striping.
Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://www-mddsp.enel.ucalgary.ca/People/adilger/
Re: [linux-lvm] lvm and 'poor mans raid' on heterogenous hard drives!
In reply to Rupa SchomakerQuoted 7 lines
Quoted 4 lines · Andreas Dilger
Swap is a bad move, since you can just add multiple swap spaces with the same priority (if you so choose) and it will do the striping for you. Likewise, you could put each of the above trees on their own drive and you would probably get better overall performance than striping.
Quoted 5 lines
This suggestion has always bothered me. Yes, if all you care about performance, setting up swap this way works fine. However, for every drive you add you increase the likelyhood that you're system will fail due to a drive failure (what happens when one of the swap slices suddenly dissapears?).
For this situation, we are considering drives failing entirely. (If you consider grown defect bad blocks appearing, then it is probably no more likely for a bad block to appear inside 1G of disk spread across two disks than it is for a bad block to appear inside 1G of disk on one disk.)
Swap devices only contain volatile data anyway.
I wouldn't mind losing the contents of my swap device. The machine will probably crash. It's the same as if the machine went down through a power outage. Not a big problem. Assuming only the swap device failed, I could just re-boot and run with a bit less swap space.
If the drive fails, I lose the contents of any filesystems on the same drive. Much more significant. The fact that the machine went down because the swap device failed pales into insignificance, because now all the data has disappeared. No-one can do any work until we have replaces disk(s), sorted out the filesystems, restored from backups etc.
If the swap had been configured on a different disk, we would have the marginal benefit of the machine probably not crashing, but the filesystems on the failed disk would still have disappeared, and we would still have the same problem.
So I spread my swap over all the fast disks, by putting them directly on to disk paritions and setting the same priority value.
Regards,
Bill
Signature · 4 lines
William H. Blunn - [email protected] - Developer Support Tao 62/63 Suttons Business Park, Earley, READING, RG6 1AZ, United Kingdom Tel: +44 118 901 2999 - Fax: +44 118 901 2963 - http://tao-group.com/
Re: [linux-lvm] lvm and 'poor mans raid' on heterogenous hard drives!
-----BEGIN PGP SIGNED MESSAGE-----
Quoted 6 lines · William Blunn
Swap devices only contain volatile data anyway.
I wouldn't mind losing the contents of my swap device. The machine will probably crash. It's the same as if the machine went down through a power outage. Not a big problem. Assuming only the swap device failed, I could just re-boot and run with a bit less swap space.
<nod> But we try to minimize the posibility of failure. Thats why many of us have a UPS hooked up. If there is a power failure we can shutdown the system cleanly on our own terms and not the power companies.
Quoted 5 lines
If the drive fails, I lose the contents of any filesystems on the same drive. Much more significant. The fact that the machine went down because the swap device failed pales into insignificance, because now all the data has disappeared. No-one can do any work until we have replaces disk(s), sorted out the filesystems, restored from backups etc.
Unless it is mirrored somehow...
- --
- -rupa
-----BEGIN PGP SIGNATURE-----
Version: 2.6.3ia
Charset: noconv
Comment: Processed by Mailcrypt 3.5.6, an Emacs/PGP interface
iQEVAwUBPHUdBnHDM4ucEopdAQGzPAf+InTVnZvUXgsMKMFxZqf8WUiFlqBwInY3 96MKgZtjtB2EvnChkVe9uUMksUGeMjarrAO9e1TMpnGzU2wjEcZySID8i+vErEYB ypG7vHT0VHLzZi+NsRlT1X7HKwLizU74TkDGE8/5H5HQN04HId9OgKeRDbi6XyLV Ms5TCfMET3q3lbT3LaYnOGjXZkOfrFsRt3z36CWaDfhOxyauZTYi3ThaCk2vs6jm PPU6PELmboJTIp7vmhv9VKSqwMMBYpgCDp20mmVZEaoRpz6ymWc0SCyyWTCrujQh RaXtolG9u8Aqj0aQvPgtKjpJKbCMspZM8Wtr55KGKmnqF+AVHkl3Tg== =6+ot
-----END PGP SIGNATURE-----