Virtual drives performance

Hello,

I'm trying to set up a fast virtualized Samba file server. I'm using Debian Sqeeze 64 bit as hypervisor.
First, I created mirrored storage in hypervisor from one 600-gig partition (yes, that's correct - I have only one drive currently), details:
sudo mdadm --detail /dev/md3
/dev/md3:

        Version : 1.2

Creation Time : Thu Jul 28 20:07:00 2011
Raid Level : raid1
Array Size : 664187352 (633.42 GiB 680.13 GB)
Used Dev Size : 664187352 (633.42 GiB 680.13 GB)
Raid Devices : 2
Total Devices : 1
Persistence : Superblock is persistent

Update Time : Thu Jul 28 22:07:10 2011
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 0
Spare Devices : 0

Name : Server:3 (local to host Server) UUID : 87184170:2d9102b1:ca16a5d7:1f23fe2e
Events : 3276

Number Major Minor RaidDevice State
0 8 23 0 active sync /dev/sdb7
1 0 0 1 removed

Partition type is Linux RAID autodetect and this drive can do 80MB/s write and 100 MB/s read seq.

QEMU PC emulator version 0.12.5 (qemu-kvm-0.12.5)

Next, I've tried following combinations with virt-manager 0.8.4 (from XML of VM):
1.on Debian VM with virtio drivers for both storage and NIC:
<disk type='block' device='disk'>
<source dev='/dev/md3'/>
<target dev='vdb' bus='virtio'/>
partition type used in guest: EXT4
result: poor performance, 9-10MB/s sequentional copy via SMB
2.on Debian VM with virtio drivers:
<disk type='block' device='disk' cache='writeback'>
<source dev='/dev/md3'/>
<target dev='vdb' bus='virtio'/>
partition type used: EXT4
result: poor performance, 10-15MB/s sequentional copy via SMB
3.Direct attached partition to FreeBSD VM without virtio support (e1000 NIC and SCSI disk):
<disk type='block' device='disk' cache='writeback'>
<source dev='/dev/md3'/>
<target dev='sdb' bus='scsi'/>
partition type used: ZFS
result: poor performance, 20-25MB/s sequentional copy via SMB
4.Direct attached whole physical disk to FreeBSD VM (/dev/sdc 2,5") partition type used in VM: ZFS
good performance, 60MB/s sequentional copy via SMB

I find that way it's not possible to direct attach PARTITION from host to VM using virt-manager, this only works for whole device (like /dev/sdc).

Anyway, any storage advices for performance?

also sprach TooMeeK <[email protected]> [2011.09.07.0025 +0200]:

This is not really necessary, presuming that you are doing what everyone is doing nowadays and assemble the RAID from the initrd or the init scripts.

You could/should be using LVM instead of partitions, IMHO.

Unfortunately, no. I have seen similar things happen. I think that the problem is Linux I/O. Your drive may be able to handle high, sustained read/write rates, but the Linux I/O scheduler, in combination with random-access reads/writes, makes these performance data inaccurate.

The only way I found to decently increase performance of drives:

- 10k and 15k drives
- external NAS or fibrechannel SAN, which do not use Linux

Yes, it's sad, and I would love to be proven wrong!

martin | http://madduck.net/ | http://two.sentenc.es/
 
"if java had true garbage collection,
 most programs would delete themselves upon execution."
                                                    -- robert sewell
 
spamtraps: [email protected]

How did you measure those figures?

To double-check sequential read throughput on the host:

# dd if=/dev/md3 of=/dev/null bs=64k count=16384 iflag=direct

The SMB results don't help narrow down a disk I/O problem. To collect comparable sequential read throughput inside the guest:

# dd if=/dev/vda of=/dev/null bs=64k count=16384 iflag=direct

Try qemu-kvm 0.15.

cache='none'

You can enable Linux AIO, which typically performs better than the default io="threads":

<driver name="qemu" type="raw" io="native"/>

Stefan

Shouldn't have you used cache=none when using virtio?
http://www.linux-kvm.org/page/Tuning_KVM
how is that performance with cache none?

Also note that when you don't specify it, I think the default is not none. Maybe it is writethrough, I don't remember.

Shouldn't have you used cache=none when using virtio?
http://www.linux-kvm.org/page/Tuning_KVM
how is that performance with cache none?

Also note that when you don't specify it, I think the default is not none. Maybe it is writethrough, I don't remember.