LVM performance questions..

I've been watching the LVM stuff for a long time. I finally gave it a try this week. I really like the tools. I think it is a great thing for Linux.. Thanks.

I have some performance questions. I was expecting to see some speed increase for a LVM stripe, but I didn't. Is there someone out there who has had more experience with this that can point me in the right direction? I found that the LVM stripe is running the same write speeds as a single disk and the same as a md RAID0 set. Sure feels like a Hardware problem, but I'm not sure.

I threw 2 9G 10k RPM disks into a VG. Then I created a 12G LV in stripe mode (at least I think I had it set right). I did a simple dd write benchmark. 25M/s (for a 1000M file). Then I tried a solo drive. 25M/s as well. I'm confused by the performance numbers -- on our Suns with SW raid we improve speed somewhat linear as we add drives
to a raid0 set. I redid the LV with stripsize of 16k and 256k. No difference in performance. Could my SCSI bus be killing me? My boot log shows this:

<6>sym53c876-0-<0,*>: FAST-20 WIDE SCSI 40.0 MB/s (50 ns, offset 16) <4>SCSI device sda: hdwr sector= 512 bytes. Sectors= 17773524 [8678 MB] [8.7 GB]

Any scsi experts out there? I did scsiinfo -i /dev/sdb and saw the 32 bit bus not set... I'm not sure if that matters or not.. Am I looking the wrong place for problems.

Inquiry command
---------------

Relative Address                   0
Wide bus 32                        0
Wide bus 16                        1
Synchronous neg.                   1
Linked Commands                    1
Command Queueing                   1

ko /home/ewhiting> cat /proc/lvm
LVM driver version 0.8i (02/10/1999)

Total: 1 VG 2 PVs 1 LV (1 LV open 1 times)

Global: 53225 bytes vmalloced IOP version: 5 2 days 17:49:02
active

VG: oracle_vg [2 PV, 1 LV/1 open] PE Size: 4096 KB
Usage [KB/PE]: 17768448 /4338 total 12582912 /3072 used 5185536
/1266 free
PVs: [AA] /dev/sdb1 8884224 /2169 6291456 /1536
2592768 /633

       [AA] /dev/sdc1              8884224 /2169     6291456 /1536    
2592768 /633   
    LV:  [AWDS2 ] oracle_lv                 12582912 /3072     1x open

eric

__________________________________________________________________
Eric T. Whiting					AMI Semiconductors   
(208) 234-6717					2300 Buckskin Road
(208) 234-6659 (fax)				Pocatello,ID 83201
[email protected]

Hi,

I would not pose myself as an absolute expert, but as far as I know, striping's biggest performance improvements come if multiple threads (or processes) are accessing the same file system (block device).

Suppose your OS does I/O in 4k units.

If your stripe set has a stripe size of larger than 4kb, than one OS logical write goes to one disk. In this case, with one process writing data there is no performance improvement to be expected.

If your stripe set has a smaller stripe size (say 1kb, or 2kb), then performance could even degrade. A 4k write will be split across two or more drives, and the entire write has to wait for the slowest disk. I won't even go into the factors involved if you have a two-deep stripe of 1kb stripe size and an OS block size of 4kb! In my view (and please correct me if I see it wrong), the stripe size must be a whole multiple of the OS block size that is going to be used in the striped LV.

With multiple processes doing I/O the story gets better. With a stripe of 2 deep, there is a 50% chance that two processes doing I/O in the same LV (file system) end up on different disks for a particular I/O. In this case, the I/O's do not have to wait for one another.

Have you tried doing I/O with more than one process? If not, could you try that as well?

++Jos

Dihydrogen Monoxide kills! 
Support a worldwide ban of Dihydrogen Monoxide. See http://www.dhmo.org