virtio disk slower than IDE?

I just tried paravirtualized virtio block devices, and my tests show that they are approximately 30% slower than emulated IDE devices. I'm guessing this isn't normal. Is this a known issue or am I likely to have mosconfigured something? I'm using 64-bit RHEL/CentOS 5 (both host and guest).

Thanks.

Gordan

Please try to change the io scheduler on the host to io scheduler, it should boost your performance back.

I presume you mean the deadline io scheduler. I tried that (kernel parameter elevator=deadline) and it made no measurable difference compared to the cfq scheduler.

Gordan

What version of kvm do you use? Is it rhel5.4?
Can you post the qemu cmdline and the perf test in the guest?

Lastly, do you use cache=wb on qemu? it's just a fun mode, we use cache=off only.

It's RHEL 5.4.

$ rpm -qa | grep -i kvm
kmod-kvm-83-105.el5_4.9
kvm-83-105.el5_4.9

Here is what is in the libvirt log:

For IDE emulation:

LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root /usr/libexec/qemu-kvm -S -M pc -m 2048 -smp 4 -name RHEL_5_x86-64 -uuid cb44b2c5-e64b-848f-77af-f8e7f02fa2ca -no-kvm-pit-reinjection -monitor pty -pidfile /var/run/libvirt/qemu//RHEL_5_x86-64.pid -boot c -drive file=/var/lib/libvirt/images/RHEL_5_x86-64.img,if=ide,index=0,boot=on -net nic,macaddr=54:52:00:5a:67:4b,vlan=0,model=e1000 -net tap,fd=15,script=,vlan=0,ifname=vnet0 -serial pty -parallel none -usb -vnc 127.0.0.1:0 -k en-gb

For virtio:

LC_ALL=C PATH=/sbin:/usr/sbin:/bin:/usr/bin HOME=/root USER=root LOGNAME=root /usr/libexec/qemu-kvm -S -M pc -m 2048 -smp 4 -name RHEL_5_x86-64 -uuid cb44b2c5-e64b-848f-77af-f8e7f02fa2ca -no-kvm-pit-reinjection -monitor pty -pidfile /var/run/libvirt/qemu//RHEL_5_x86-64.pid -boot c -drive file=/var/lib/libvirt/images/CentOS_5_x86-64.img,if=virtio,index=0,boot=on -net nic,macaddr=54:52:00:5a:67:4b,vlan=0,model=e1000 -net tap,fd=15,script=,vlan=0,ifname=vnet0 -serial pty -parallel none -usb -vnc 127.0.0.1:0 -k en-gb

The test is building the Linux kernel (only taking the second run to give the test the benefit of local cache):

make clean; make -j8 all; make clean; sync; time make -j8 all

This takes about 10 minutes with IDE disk emulation and about 13 minutes with virtio. I ran the tests multiple time with most non-essential services on the host switched off (including cron/atd), and the guest in single-user mode to reduce the "noise" in the test to the minimum, and the results are pretty consistent, with virtio being about 30% behind.

I don't see the option being set in the logs, so I'd guess it's whatever qemu-kvm defaults to.

Gordan

I'd expect for an observed 30% wall clock time difference of an operation as complex as a kernel build the base i/o throughput disparity is substantially greater. Did you try a more simple/regular load, eg: a streaming dd read of various block sizes from guest raw disk devices? This is also considerably easier to debug vs. the complex i/o load generated by a build.

One way to chop up the problem space is using blktrace on the host to observe both the i/o patterns coming out of qemu and the host's response to them in terms of turn around time. I expect you'll see somewhat different nature requests generated by qemu w/r/t blocking and number of threads serving virtio_blk requests relative to ide but the host response should be essentially the same in terms of data returned per unit time.

If the host looks to be turning around i/o request with similar latency in both cases, the problem would be lower frequency of requests generated by qemu in the case of
virtio_blk. Here it would be useful to know the host
load generated by the guest for both cases.

-john

[email protected]

You can set this through libvirt by putting an element such as the following within your <disk> element:

      <driver name='qemu' type='qcow2' cache='none'/>

(Setting the type is preferred to avoid security issues wherein a guest writes an arbitrary qcow2 header to the beginning of a raw disk, reboots and allows qemu's autodetection to decide that this formerly-raw disk should now be treated as a delta against a file they otherwise might not have access to read; as such, it's particularly important if you intend that the type be raw).

It's not needed on rhel5.4 qemu - we have cache=none as a default