LVM performance
Hi all,
All right. I've got LVM installed and running on 2 30GB IDE drives, a VG on each, and the partitions divided just about evenly between them. This is on a 1GHz Athlon with 640MB of RAM. I am running Debian/sid with lvm 1.0.1rc4 and kernel 2.4.13.
A few things I have noticed in the week or so that I have been using LVM. First, the drive activity light stays lit all the time. I'm not terribly worried about it, since things seem to be behaving normally, but I figured I would ask.
The second, and far more disturbing thing is that any heavy accesses to the filesystem (copying large files, scp-ing files mastering CDs, anything with a lot of IO brings my machine to its knees. Case in point. When I started this message, I was mastering a CD using xcdroast, which had called mkisofs. Not only did the mastering take (I'm guessing here, since I didn't note the time) about 10 minutes, the load average jumped, at one point, to 8.23, and generally hovered around the 6-to-7 range. Copying large files from filesystem to filesystem on the machine yields similar results. Originally, I had the following configuration:
hda 30GB HD
hdb 30GB HD
hdc DVD-ROM
hdd CD-RW
A friend suggested moving the HD on hdb to the other channel, so now the drives are on hda and hdc, the dvd is on hdb and the cdrw is on hdd.
Can anyone give me any ideas as to why the machine gets beaten about so much during IO operations and more importantly how can I minimize the impact.
Regards,
Signature · 9 lines
--Brad ============================================================================ Bradley M. Alexander, CISSP | Co-Chairman, Beowulf System Admin/Security Specialist | NoVALUG/DCLUG Security SIG Debian/GNU Linux Developer | [email protected] | [email protected] ============================================================================ Never sign a contract which contains the phrases "sort of," "kind of," or "and stuff."
[big snip]
Quoted 9 lines
Originally, I had the following configuration:
hda 30GB HD
hdb 30GB HD
hdc DVD-ROM
hdd CD-RWA friend suggested moving the HD on hdb to the other channel, so now the drives are on hda and hdc, the dvd is on hdb and the cdrw is on hdd.
thats dead right; ide can't simultaneously write to the master and slave on the same controller. Also, the pair of drives uses the controller circuitry
of the master, so it always pays to make the master the most modern one.
Quoted 3 lines
Can anyone give me any ideas as to why the machine gets beaten about so much during IO operations and more importantly how can I minimize the impact.
Dunno, if it was because the fs is striped across those drives then splitting them across controllers would have made it go away.
I've seen no performance problems at all and really thrashed an LVM-root machine for test purposes while working on a movie. It took it well, performance-wise. (reliability is another issue; never go LVM-root... but thats just my 2 cents, YMMV).
Quoted 5 lines · Steve Wray
thats dead right; ide can't simultaneously write to the master and slave on the same controller. Also, the pair of drives uses the controller circuitry
of the master, so it always pays to make the master the most modern one.
Since both controllers have a drive master and a cd slave, this shouldn't matter, per se.
Quoted 7 lines
Quoted 4 lines
Can anyone give me any ideas as to why the machine gets beaten about so much during IO operations and more importantly how can I minimize the impact.
Dunno, if it was because the fs is striped across those drives then splitting them across controllers would have made it go away.
Nothing should be striped across the drives. I have two PVs and made two VGs, one on each drive. The way it works out, the data I am mastering is on vg01 (hdc), and the ISO is being created there as well. the CD write actually happens from hdc -> hdd.
Quoted 4 lines
I've seen no performance problems at all and really thrashed an LVM-root machine for test purposes while working on a movie. It took it well, performance-wise. (reliability is another issue; never go LVM-root... but thats just my 2 cents, YMMV).
Yeah, I wasn't brave enough to go all out...And with the problems I was having, I'm glad I made that decision. :)
Quoted 7 lines
_______________________________________________
linux-lvm mailing list
[email protected]
http://lists.sistina.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://www.sistina.com/lvm/Pages/howto.html
Signature · 11 lines
--Brad ============================================================================ Bradley M. Alexander, CISSP | Co-Chairman, Beowulf System Admin/Security Specialist | NoVALUG/DCLUG Security SIG Debian/GNU Linux Developer | [email protected] | [email protected] ============================================================================ You start with a bag full of luck and an empty bag of experience. The trick is to fill the bag of experience before you empty the bag of luck. --Rules of the Air, #16
Quoted 11 lines
From: [email protected] [mailto:[email protected]]On Behalf Of Bradley M Alexander
Quoted 5 lines · Steve Wray
thats dead right; ide can't simultaneously write to the master and slave on the same controller. Also, the pair of drives uses the controller circuitry
of the master, so it always pays to make the master the most modern one.Since both controllers have a drive master and a cd slave, this shouldn't matter, per se.
yeah but I thought I'd mention it, for completeness :)
Quoted 11 lines
Quoted 1 line
Quoted 1 line
Can anyone give me any ideas as to why the machine gets
beaten about so
Quoted 5 lines
Quoted 2 lines
much during IO operations and more importantly how can I minimize the impact.
Dunno, if it was because the fs is striped across those drives then splitting them across controllers would have made it go away.
Nothing should be striped across the drives. I have two PVs and made two VGs, one on each drive. The way it works out, the data I am mastering is on
yup you have to tell it to stripe, as I recall. Never used that feature tho.
Quoted 7 lines
Quoted 4 lines
I've seen no performance problems at all and really thrashed an LVM-root machine for test purposes while working on a movie. It took it well, performance-wise. (reliability is another issue; never go LVM-root... but thats just my 2 cents, YMMV).
Yeah, I wasn't brave enough to go all out...And with the problems I was having, I'm glad I made that decision. :)
I think the worst part is the lack of backward compatibility at some stages of LVM. With root on LVM it can be a bit dangerous to upgrade! Or at least it was...
-- Steve Wray <[email protected]>
Quoted 10 lines
Quoted 10 lines
From: [email protected] [mailto:[email protected]]On Behalf Of Bradley M Alexander
Quoted 6 lines · Steve Wray
thats dead right; ide can't simultaneously write to the master and slave on the same controller. Also, the pair of drives uses the controller circuitry
of the master, so it always pays to make the master the most modern one.
Quoted 5 lines
Quoted 2 lines
Since both controllers have a drive master and a cd slave, this shouldn't matter, per se.
yeah but I thought I'd mention it, for completeness :)
Dependong on the bios, for example, there still could be contention on the SB chipset for access to the controllers. Point is that there can be multiple bottlenecks.
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
Quoted 1 line
Can anyone give me any ideas as to why the machine gets
beaten about so
Quoted 5 lines
Quoted 2 lines
much during IO operations and more importantly how can I minimize the impact.
Dunno, if it was because the fs is striped across those drives then splitting them across controllers would have made it go away.
Quoted 6 lines
Quoted 3 lines
Nothing should be striped across the drives. I have two PVs and made two VGs, one on each drive. The way it works out, the data I am mastering is on
yup you have to tell it to stripe, as I recall. Never used that feature tho.
I'm using LVM with several striped volumes. Hardware is Tekram 160MB scsi + pair of 80MB drives. The striping works rather
nicely: sustained I/O is high for large copies and latency is low for reads. The file system uses a 4KB page w/ the inodes reduced to a reasonable value (e.g., mkfs.ext2 -b4096 -i10240).
If you are using the default of 1block / page then it can hammer a system due to 2ndary access and higher write rate.
Quoted 11 lines
Quoted 7 lines
Quoted 4 lines
I've seen no performance problems at all and really thrashed an LVM-root machine for test purposes while working on a movie. It took it well, performance-wise. (reliability is another issue; never go LVM-root... but thats just my 2 cents, YMMV).
Yeah, I wasn't brave enough to go all out...And with the problems I was having, I'm glad I made that decision. :)
I think the worst part is the lack of backward compatibility at some stages of LVM. With root on LVM it can be a bit dangerous to upgrade! Or at least it was...
<broken-record>
This can also be a real pain to fix if something goes wrong. If the lvm is --prefix-ed to, say, /lvm and the root volume is on a paritition then you can fix most things w/o having to play with rescue file systems.
</broken-record>
--
Steven Lembark 2930 W. Palmer
Workhorse Computing Chicago, IL 60647
+1 800 762 1582
Quoted 1 line · Bradley M Alexander
the CD write actually happens from hdc -> hdd.
This might be *a* problem. As others have said, IDE handles bus
contention either very poorly or not at all. Since you have all your I/O on one bus, you may be seeing a problem there.
For IDE, I generally favor putting HDs on one bus, and CDs/tapes on the
other. Actually, I favor putting a single device on each bus. Actually, I favor SCSI. :-)
Signature · 4 lines
Ben Scott <[email protected]> | The opinions expressed in this message are those of the author and do not | | necessarily represent the views or policy of any other person, entity or | | organization. All information is provided without warranty of any kind. |
-- Benjamin Scott <[email protected]>
Quoted 2 lines
Actually,
I favor SCSI. :-)
Me too, but more scsi-ish behavior can be arrived at by tinkering with hdparm. Mainly the delayed write, buffer size can help enormously.
--
Steven Lembark 2930 W. Palmer
Workhorse Computing Chicago, IL 60647
+1 800 762 1582
I hope this isn't stating the obvious, but I asume you have used hdparm to adjust the default setting on your drives appropriatly (i.e. dma). I have 4 disks striped and get average sequential throuput of at least 40mb/s and 35mb/s (dd to /dev/null) on only 2 disks.
cpu is approx 60% system (666mhz system).
If I have the disks untuned I get %100 cpu, machine "stuttering" and throuput 10 times less.
Duncan
Quoted 35 lines · you
Hi all,
All right. I've got LVM installed and running on 2 30GB IDE drives, a VG on each, and the partitions divided just about evenly between them. This is on a 1GHz Athlon with 640MB of RAM. I am running Debian/sid with lvm 1.0.1rc4 and kernel 2.4.13.
A few things I have noticed in the week or so that I have been using LVM. First, the drive activity light stays lit all the time. I'm not terribly worried about it, since things seem to be behaving normally, but I figured I would ask.
The second, and far more disturbing thing is that any heavy accesses to the filesystem (copying large files, scp-ing files mastering CDs, anything with a lot of IO brings my machine to its knees. Case in point. When I started this message, I was mastering a CD using xcdroast, which had called mkisofs. Not only did the mastering take (I'm guessing here, since I didn't note the time) about 10 minutes, the load average jumped, at one point, to 8.23, and generally hovered around the 6-to-7 range. Copying large files from filesystem to filesystem on the machine yields similar results. Originally, I had the following configuration:
hda 30GB HD
hdb 30GB HD
hdc DVD-ROM
hdd CD-RWA friend suggested moving the HD on hdb to the other channel, so now the drives are on hda and hdc, the dvd is on hdb and the cdrw is on hdd.
Can anyone give me any ideas as to why the machine gets beaten about so much during IO operations and more importantly how can I minimize the impact.
Regards,