Skip to contentSkip to conversation

kvm-14 release

ANNOUNCE Avi Kivity 10 participants 31 messages Feb 19, 2007 – Mar 5, 2007 23 messages not in the archive KVM

lore
Share & tools
lore

Avi Kivity

author
#0 lore raw

The big news here is the update to qemu 0.9.0.

Changes from kvm-13:
- qemu 0.9.0
- too many goodies to list
- kvm can no longer share qemu's bios on Intel hosts due to real mode
trouble. use the supplied bios.
- migration now based on Anthony Liguori's live migration patches (Uri Lublin)
- currently, only non-live migration is supported under kvm
- handle smi on host on AMD hosts (Joerg Roedel)
- random fixes

Note that if you use the modules from Linux 2.6.20, you need to use kvm-12. You can use kvm-14 with Linux 2.6.20, provided you use the external module included in kvm-14.

http://kvm.qumranet.com

Signature · 8 lines
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): 45D98390.6060001-atKUWr5tajBWk0Htik3J/[email protected]

#2 lore raw
Quoted 13 lines · Avi Kivity

The big news here is the update to qemu 0.9.0.

Changes from kvm-13:
- qemu 0.9.0
- too many goodies to list
- kvm can no longer share qemu's bios on Intel hosts due to real mode
trouble. use the supplied bios.
- migration now based on Anthony Liguori's live migration patches (Uri Lublin)
- currently, only non-live migration is supported under kvm
- handle smi on host on AMD hosts (Joerg Roedel)
- random fixes

Quoted 3 lines

Note that if you use the modules from Linux 2.6.20, you need to use kvm-12. You can use kvm-14 with Linux 2.6.20, provided you use the external module included in kvm-14.

Is there back-compatability in the reverse direction ? ie, will the new kmod from kvm-14 work with the older kvm-12 userspace ?

Regards,
Dan.

Signature · 10 lines
|=- Red Hat, Engineering, Emerging Technologies, Boston.  +1 978 392 2496 -=|
|=-           Perl modules: http://search.cpan.org/~danberr/              -=|
|=-               Projects: http://freshmeat.net/~danielpb/               -=|
|=-  GnuPG: 7D3B9505   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505  -=| 

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#4 lore raw
Quoted 8 lines · Daniel P. Berrange
Quoted 4 lines

Note that if you use the modules from Linux 2.6.20, you need to use kvm-12. You can use kvm-14 with Linux 2.6.20, provided you use the external module included in kvm-14.

Is there back-compatability in the reverse direction ? ie, will the new kmod from kvm-14 work with the older kvm-12 userspace ?

No.

Signature · 8 lines
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): 45D9A464.8040406-atKUWr5tajBWk0Htik3J/[email protected]

#6 lore raw
Quoted 11 lines · Avi Kivity
Quoted 8 lines · Daniel P. Berrange
Quoted 4 lines

Note that if you use the modules from Linux 2.6.20, you need to use kvm-12. You can use kvm-14 with Linux 2.6.20, provided you use the external module included in kvm-14.

Is there back-compatability in the reverse direction ? ie, will the new kmod from kvm-14 work with the older kvm-12 userspace ?

No.

It's traditional for the mainline kernel to maintain backward compatibility with existing userspace ABIs (e.g. syscalls).

The idea of a userspace application breaking because of a mainline kernel upgrade is not acceptable to users and also pretty much unsupportable by distros.

- James

Signature · 8 lines
James Morris
<jmorris-gx6/[email protected]>

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
author
#7 lore raw
Quoted 7 lines · James Morris

It's traditional for the mainline kernel to maintain backward compatibility with existing userspace ABIs (e.g. syscalls).

The idea of a userspace application breaking because of a mainline kernel upgrade is not acceptable to users and also pretty much unsupportable by distros.

That is of course very true, and it is a goal of kvm to have a stable userspace interface. I plan to target 2.6.21 as the stable ABI release.

Signature · 8 lines
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): 45D9C57D.9030203-atKUWr5tajBWk0Htik3J/[email protected]

author
#9 lore raw
Quoted 4 lines · Avi Kivity
Quoted 3 lines · James Morris

It's traditional for the mainline kernel to maintain backward compatibility with existing userspace ABIs (e.g. syscalls).

Quoted 8 lines
Quoted 4 lines

The idea of a userspace application breaking because of a mainline kernel upgrade is not acceptable to users and also pretty much unsupportable by distros.

That is of course very true, and it is a goal of kvm to have a stable userspace interface. I plan to target 2.6.21 as the stable ABI release.

As the alert reader will have guessed, this is now postponed to 2.6.22. I will provide a nonintrusive patch backporting the stable ABI to 2.6.21 if desired.

Signature · 8 lines
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
#10 lore raw
In reply to James Morris
Quoted 8 lines · James Morris
Quoted 6 lines · Avi Kivity
Quoted 5 lines · Daniel P. Berrange
Quoted 4 lines

Note that if you use the modules from Linux 2.6.20, you need to use kvm-12. You can use kvm-14 with Linux 2.6.20, provided you use the external module included in kvm-14.

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

Is there back-compatability in the reverse direction ? ie, will the new kmod from kvm-14 work with the older kvm-12 userspace ?

No.

It's traditional for the mainline kernel to maintain backward compatibility with existing userspace ABIs (e.g. syscalls).

Stable ABIs are hard, and this is CONFIG_EXPERIMENTAL after all. I don't think KVM needs ABI stability until CONFIG_EXPERIMENTAL is removed (of course, sooner is always nice if it doesn't create problems).

Rusty.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): 1171933567.8218.53.camel-bi+AKbBUZKY6gyzm1THtWbp2dZbC/[email protected]

#12 lore raw
Quoted 3 lines · Rusty Russell

Stable ABIs are hard, and this is CONFIG_EXPERIMENTAL after all. I don't think KVM needs ABI stability until CONFIG_EXPERIMENTAL is removed (of course, sooner is always nice if it doesn't create problems).

FC7 will ship with kvm and 2.6.21, so changing the ABI after that will cause problems for real users and distro maintainers. We already have enough of this to deal with due to Xen.

- James

Signature · 8 lines
James Morris
<jmorris-gx6/[email protected]>

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
#13 lore raw
In reply to a message not in the archive
Quoted 13 lines · On Monday 19 February 2007 09:01:36 Avi Kivity wrote:

The big news here is the update to qemu 0.9.0.

Changes from kvm-13:
- qemu 0.9.0
- too many goodies to list
- kvm can no longer share qemu's bios on Intel hosts due to real mode
trouble. use the supplied bios.
- migration now based on Anthony Liguori's live migration patches (Uri Lublin)
- currently, only non-live migration is supported under kvm
- handle smi on host on AMD hosts (Joerg Roedel)
- random fixes

Quoted 3 lines

Note that if you use the modules from Linux 2.6.20, you need to use kvm-12. You can use kvm-14 with Linux 2.6.20, provided you use the external module included in kvm-14.

How are you guys in general dealing with the gcc-3.x requirement? Most distros already use gcc-4.x.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): 200702191237.39291.ahasenack-y7mWNqJcIDpfJ/[email protected]

author
#15 lore raw
Quoted 3 lines · Andreas Hasenack

How are you guys in general dealing with the gcc-3.x requirement? Most distros already use gcc-4.x.

If you're not planning to use the -no-kvm option, you can use gcc 4. Otherwise, you can use the gcc 3 compat packages that most distros provide.

Note that only qemu needs gcc 3. The kernel module and libkvm.a can and should use the system compiler.

Signature · 8 lines
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
#16 lore raw
In reply to Andreas Hasenack
Quoted 4 lines · Andreas Hasenack

How are you guys in general dealing with the gcc-3.x requirement? Most distros already use gcc-4.x.

gcc-3.x for qemu? Here is how you can install gcc-3.x without breaking gcc-4.x:

http://en.opensuse.org/Qemu_with_kqemu_kernel_module_support

you can then configure qemu with the appropriate options.

Omar Khan

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#18 lore raw
Quoted 7 lines · Omar Khan
Quoted 4 lines · Andreas Hasenack

How are you guys in general dealing with the gcc-3.x requirement? Most distros already use gcc-4.x.

Quoted 7 lines

gcc-3.x for qemu? Here is how you can install gcc-3.x without breaking gcc-4.x:

http://en.opensuse.org/Qemu_with_kqemu_kernel_module_support

you can then configure qemu with the appropriate options.

Some distros provide a gcc 3 package in addition to gcc 4. For example, on Fedora Core 6, it is called compat-gcc-34.

Signature · 8 lines
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
#19 lore raw
In reply to a message not in the archive
Quoted 13 lines · Avi Kivity

The big news here is the update to qemu 0.9.0.

Changes from kvm-13:
- qemu 0.9.0
- too many goodies to list
- kvm can no longer share qemu's bios on Intel hosts due to real mode
trouble. use the supplied bios.
- migration now based on Anthony Liguori's live migration patches (Uri Lublin)
- currently, only non-live migration is supported under kvm
- handle smi on host on AMD hosts (Joerg Roedel)
- random fixes

Thanks a lot for your work. I have just tried it.

The good news is that kvm is now able to boot a Hurd system, this what no the case with kvm-12.

The bad news is that kvm-14 seems to be a lot slower than kvm-12 + modules from kernel 2.6.20. This is the case with a GNU/kFreeBSD guest. kvm-12 was about 1.5 time faster than qemu + kqemu. kvm-20 is slower than qemu without kqemu...

Does anybody have an idea about this performance regression?

Signature · 10 lines
  .''`.  Aurelien Jarno	            | GPG: 1024D/F1BCDB73
 : :' :  Debian developer           | Electrical Engineer
 `. `'   [email protected]         | [email protected]
   `-    people.debian.org/~aurel32 | www.aurel32.net

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#21 lore raw
Quoted 7 lines · Aurelien Jarno

The bad news is that kvm-14 seems to be a lot slower than kvm-12 + modules from kernel 2.6.20. This is the case with a GNU/kFreeBSD guest. kvm-12 was about 1.5 time faster than qemu + kqemu. kvm-20 is slower than qemu without kqemu...

Does anybody have an idea about this performance regression?

What is your workload? How are you measuring performance?

Signature · 8 lines
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): 45DA9FFA.2020009-atKUWr5tajBWk0Htik3J/[email protected]

#23 lore raw
Quoted 10 lines · Avi Kivity
Quoted 7 lines · Aurelien Jarno

The bad news is that kvm-14 seems to be a lot slower than kvm-12 + modules from kernel 2.6.20. This is the case with a GNU/kFreeBSD guest. kvm-12 was about 1.5 time faster than qemu + kqemu. kvm-20 is slower than qemu without kqemu...

Does anybody have an idea about this performance regression?

What is your workload? How are you measuring performance?

Sorry to answer only now, it took me some time to do some more measurements and have some numbers. I am simply building a Debian package (simulpic) and measuring the build time. Basically the command is:

apt-get source simulpic
cd simulpic-2005-1-28
time debuild -uc us

It surely not a performance index, but I guess it is ok to compare performances between version. It is also quite representative of my use of the machine.

The guest is Debian GNU/kFreeBSD amd64 (ie FreeBSD kernel + GNU libc). It is accessed via ssh, and kvm is started with -nographic, so there is no influence of xorg.

I am doing my tests on an Athlon X2 3800+ machine, running a 2.6.20 kernel. During all my tests, the machine is not loaded with other tasks (except systems tasks), so qemu or kvm have a full core available. Top shows that the core is used between 95 and 100% during the whole build in all cases.

The tests I have made are presented below. In all cases I have verified that the real time correspond to the time of my wall clock, it is correct in all case given the resolution of my wall clock (1 s):

qemu
----

real    3m16.626s
user    2m22.654s
sys     0m41.738s

qemu + kqemu
------------

real    0m51.529s
user    0m11.775s
sys     0m36.215s

kvm 12 + modules from kernel 2.6.20
-----------------------------------

real    0m30.635s
user    0m16.357s
sys     0m8.511s

kvm 12
------

real    0m25.357s
user    0m16.259s
sys     0m6.496s

kvm 13
------

real    0m23.415s
user    0m15.177s
sys     0m5.811s

kvm 14
------

real    7m47.310s
user    5m17.359s
sys     2m3.184s

Using kvm 14, the system is clearly not responsive at all. You can see that without running a benchmark.

Also running Hurd under kvm shows the same problem, though I can't make such measurements, because it does not run under kvm < 14. Note however that the ping time of the guest machine running Hurd during I/O increases from a few milliseconds to a dozen of seconds.

Signature · 10 lines
  .''`.  Aurelien Jarno	            | GPG: 1024D/F1BCDB73
 : :' :  Debian developer           | Electrical Engineer
 `. `'   [email protected]         | [email protected]
   `-    people.debian.org/~aurel32 | www.aurel32.net

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#25 lore raw
Quoted 19 lines · Aurelien Jarno
Quoted 7 lines · Avi Kivity
Quoted 5 lines · Aurelien Jarno

The bad news is that kvm-14 seems to be a lot slower than kvm-12 + modules from kernel 2.6.20. This is the case with a GNU/kFreeBSD guest. kvm-12 was about 1.5 time faster than qemu + kqemu. kvm-20 is slower than qemu without kqemu...

Quoted 5 lines
Quoted 3 lines

Does anybody have an idea about this performance regression?

What is your workload? How are you measuring performance?

Sorry to answer only now, it took me some time to do some more measurements and have some numbers. I am simply building a Debian package (simulpic) and measuring the build time. Basically the command is:

Quoted 8 lines

apt-get source simulpic
cd simulpic-2005-1-28
time debuild -uc us

It surely not a performance index, but I guess it is ok to compare performances between version. It is also quite representative of my use of the machine.

Real workloads (likr this) are more important than synthetic benchmarks.

Quoted 14 lines

The guest is Debian GNU/kFreeBSD amd64 (ie FreeBSD kernel + GNU libc). It is accessed via ssh, and kvm is started with -nographic, so there is no influence of xorg.

I am doing my tests on an Athlon X2 3800+ machine, running a 2.6.20 kernel. During all my tests, the machine is not loaded with other tasks (except systems tasks), so qemu or kvm have a full core available. Top shows that the core is used between 95 and 100% during the whole build in all cases.

The tests I have made are presented below. In all cases I have verified that the real time correspond to the time of my wall clock, it is correct in all case given the resolution of my wall clock (1 s):

Quoted 6 lines

qemu
----

real    3m16.626s
user    2m22.654s
sys     0m41.738s

Is this qemu 0.8.2 or qemu 0.9.0?

Quoted 18 lines

qemu + kqemu
------------

real    0m51.529s
user    0m11.775s
sys     0m36.215s

kvm 12 + modules from kernel 2.6.20
-----------------------------------

real    0m30.635s
user    0m16.357s
sys     0m8.511s

kvm 12
------

real    0m25.357s
user    0m16.259s
sys     0m6.496s
Quoted 6 lines

kvm 13
------

real    0m23.415s
user    0m15.177s
sys     0m5.811s

So far so good. Steady improvement. The low system time indicates a lot of I/O and inefficiency in the qemu device emulation (guest time is charged as system time).

Quoted 10 lines

kvm 14
------

real    7m47.310s
user    5m17.359s
sys     2m3.184s

Using kvm 14, the system is clearly not responsive at all. You can see that without running a benchmark.

kvm-14 is mostly qemu 0.9.0. Do you get the same results with kvm-14 -no-kvm?

What is your disk image file format, or are you using a partition?

Do the results change (on kvm-14) if you pin the guest to a core with 'taskset 1 qemu ...'

Thank you for taking the time to do real measurements and report the results clearly. That makes it possible (I hope) to find the cause and fix it.

Signature · 8 lines
Do not meddle in the internals of kernels, for they are subtle and quick to panic.


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): 45DBFD6E.2060507-atKUWr5tajBWk0Htik3J/[email protected]

#27 lore raw
Quoted 44 lines · Avi Kivity
Quoted 18 lines · Aurelien Jarno
Quoted 7 lines · Avi Kivity
Quoted 5 lines · Aurelien Jarno

The bad news is that kvm-14 seems to be a lot slower than kvm-12 + modules from kernel 2.6.20. This is the case with a GNU/kFreeBSD guest. kvm-12 was about 1.5 time faster than qemu + kqemu. kvm-20 is slower than qemu without kqemu...

Quoted 5 lines
Quoted 3 lines

Does anybody have an idea about this performance regression?

What is your workload? How are you measuring performance?

Sorry to answer only now, it took me some time to do some more measurements and have some numbers. I am simply building a Debian package (simulpic) and measuring the build time. Basically the command is:

Quoted 8 lines

apt-get source simulpic
cd simulpic-2005-1-28
time debuild -uc us

It surely not a performance index, but I guess it is ok to compare performances between version. It is also quite representative of my use of the machine.

Real workloads (likr this) are more important than synthetic benchmarks.

Quoted 14 lines

The guest is Debian GNU/kFreeBSD amd64 (ie FreeBSD kernel + GNU libc). It is accessed via ssh, and kvm is started with -nographic, so there is no influence of xorg.

I am doing my tests on an Athlon X2 3800+ machine, running a 2.6.20 kernel. During all my tests, the machine is not loaded with other tasks (except systems tasks), so qemu or kvm have a full core available. Top shows that the core is used between 95 and 100% during the whole build in all cases.

The tests I have made are presented below. In all cases I have verified that the real time correspond to the time of my wall clock, it is correct in all case given the resolution of my wall clock (1 s):

Quoted 8 lines
Quoted 6 lines

qemu
----

real    3m16.626s
user    2m22.654s
sys     0m41.738s

Is this qemu 0.8.2 or qemu 0.9.0?

It's qemu 0.9.0 + kqemu 1.3.0pre11. kqemu is enabled in user mode only (kernel mode does not work well on amd64)

Quoted 36 lines
Quoted 18 lines

qemu + kqemu
------------

real    0m51.529s
user    0m11.775s
sys     0m36.215s

kvm 12 + modules from kernel 2.6.20
-----------------------------------

real    0m30.635s
user    0m16.357s
sys     0m8.511s

kvm 12
------

real    0m25.357s
user    0m16.259s
sys     0m6.496s
Quoted 6 lines

kvm 13
------

real    0m23.415s
user    0m15.177s
sys     0m5.811s

So far so good. Steady improvement. The low system time indicates a lot of I/O and inefficiency in the qemu device emulation (guest time is charged as system time).

Quoted 7 lines

kvm 14
------

real    7m47.310s
user    5m17.359s
sys     2m3.184s
Quoted 6 lines
Quoted 3 lines

Using kvm 14, the system is clearly not responsive at all. You can see that without running a benchmark.

kvm-14 is mostly qemu 0.9.0. Do you get the same results with kvm-14 -no-kvm?

Here are the results:

real    3m45.459s
user    2m48.581s
sys     0m48.585s

It's a bit slower than qemu 0.9.0, but a lot faster than kvm 0.9.0 with kvm enable.

Quoted 1 line

What is your disk image file format, or are you using a partition?

I am using a raw image file on an ext3 partition.

Quoted 2 lines

Do the results change (on kvm-14) if you pin the guest to a core with 'taskset 1 qemu ...'

Bingo. It now works even faster than kvm-13!

real    0m22.307s
user    0m13.935s
sys     0m4.720
Quoted 3 lines

Thank you for taking the time to do real measurements and report the results clearly. That makes it possible (I hope) to find the cause and fix it.

Thanks for your help! Do you think this problem is fixable? On my final machine, I have a dozen of qemu/kvm running, and when I start them I don't know how they will be used, and so how to pin them on the two cores.

Signature · 10 lines
  .''`.  Aurelien Jarno	            | GPG: 1024D/F1BCDB73
 : :' :  Debian developer           | Electrical Engineer
 `. `'   [email protected]         | [email protected]
   `-    people.debian.org/~aurel32 | www.aurel32.net

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#29 lore raw
Quoted 11 lines · Aurelien Jarno
Quoted 3 lines

What is your disk image file format, or are you using a partition?

I am using a raw image file on an ext3 partition.

Quoted 3 lines

Do the results change (on kvm-14) if you pin the guest to a core with 'taskset 1 qemu ...'

Quoted 7 lines

Bingo. It now works even faster than kvm-13!

real    0m22.307s
user    0m13.935s
sys     0m4.720

I'm guessing this is due to the glibc aio implementation, which uses threads instead of true aio. The threads may cause the vcpu to migrate frequently from one code to another.

There are two possible solutions:

- use native aio from http://www.bullopensource.org/posix/. I think
the aio signal patches are not yet in, so this may not work.
- teach the scheduler about the cost of migrating vcpus

The first approach will solve itself eventually, though slowly if the current slow rate of aio merging continues. We'll have to do the second.

Quoted 10 lines
Quoted 5 lines

Thank you for taking the time to do real measurements and report the results clearly. That makes it possible (I hope) to find the cause and fix it.

Thanks for your help! Do you think this problem is fixable? On my final machine, I have a dozen of qemu/kvm running, and when I start them I don't know how they will be used, and so how to pin them on the two cores.

For a temporary solution, I'd just pick a random core in a startup script and trust to probability. Hopefully the scheduler will get fixed to handle vcpus soon.

Signature · 8 lines
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): 45DC54B5.9080608-atKUWr5tajBWk0Htik3J/[email protected]

#31 lore raw
Quoted 22 lines · Avi Kivity
Quoted 6 lines · Aurelien Jarno
Quoted 3 lines

What is your disk image file format, or are you using a partition?

I am using a raw image file on an ext3 partition.

Quoted 10 lines
Quoted 3 lines

Do the results change (on kvm-14) if you pin the guest to a core with 'taskset 1 qemu ...'

Bingo. It now works even faster than kvm-13!

real    0m22.307s
user    0m13.935s
sys     0m4.720

I'm guessing this is due to the glibc aio implementation, which uses threads instead of true aio. The threads may cause the vcpu to migrate frequently from one code to another.

Quoted 8 lines

There are two possible solutions:

- use native aio from http://www.bullopensource.org/posix/. I think
the aio signal patches are not yet in, so this may not work.
- teach the scheduler about the cost of migrating vcpus
The first approach will solve itself eventually, though slowly if the current slow rate of aio merging continues. We'll have to do the second.

if you prefer the first one, Sébastien will release very soon aio patches for 2.6.20 with an up-to-date libposix-aio.

[advertising] Keep an eye on the bullopensource website. [/advertising] :-P

Regards,
Laurent

Signature · 3 lines
------------- Laurent.Vivier-6ktuUTfB/[email protected]  --------------
       "Any sufficiently advanced technology is
  indistinguishable from magic." - Arthur C. Clarke

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

A message that is not in the archive (sent off-list or not yet received): 45DC5D4E.5000300-6ktuUTfB/[email protected]

#33 lore raw
Quoted 39 lines · Laurent Vivier
Quoted 27 lines · Avi Kivity
Quoted 13 lines · Aurelien Jarno
Quoted 5 lines

What is your disk image file format, or are you using a partition?

I am using a raw image file on an ext3 partition.

Quoted 4 lines

Do the results change (on kvm-14) if you pin the guest to a core with 'taskset 1 qemu ...'

Quoted 8 lines

Bingo. It now works even faster than kvm-13!

real    0m22.307s
user    0m13.935s
sys     0m4.720

I'm guessing this is due to the glibc aio implementation, which uses threads instead of true aio. The threads may cause the vcpu to migrate frequently from one code to another.

Quoted 9 lines

There are two possible solutions:

- use native aio from http://www.bullopensource.org/posix/. I think
the aio signal patches are not yet in, so this may not work.
- teach the scheduler about the cost of migrating vcpus
The first approach will solve itself eventually, though slowly if the current slow rate of aio merging continues. We'll have to do the second.

Quoted 3 lines

if you prefer the first one, Sébastien will release very soon aio patches for 2.6.20 with an up-to-date libposix-aio.

Hi Laurent,

I gave that a shot a little bit ago. Ran into two problems.

1) Couldn't avoid linking to -lrt as QEMU uses time functions from it.
2) While I could get things compiling (with patches), QEMU would SEGV almost immediately.

Could you guys maybe give compiling QEMU w/libposix-aio a shot? I'm really interested to see if it makes a difference.

Regards,

Anthony Liguori

Quoted 15 lines

[advertising] Keep an eye on the bullopensource website. [/advertising] :-P

Regards,
Laurent

------------------------------------------------------------------------

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
------------------------------------------------------------------------

Quoted 5 lines

_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): [email protected]

#35 lore raw
Quoted 39 lines · Anthony Liguori
Quoted 38 lines · Laurent Vivier
Quoted 36 lines · Avi Kivity
Quoted 13 lines · Aurelien Jarno
Quoted 5 lines

What is your disk image file format, or are you using a partition?

I am using a raw image file on an ext3 partition.

Quoted 4 lines

Do the results change (on kvm-14) if you pin the guest to a core with 'taskset 1 qemu ...'

Quoted 8 lines

Bingo. It now works even faster than kvm-13!

real    0m22.307s
user    0m13.935s
sys     0m4.720

I'm guessing this is due to the glibc aio implementation, which uses threads instead of true aio. The threads may cause the vcpu to migrate frequently from one code to another.

There are two possible solutions:

- use native aio from http://www.bullopensource.org/posix/. I think
the aio signal patches are not yet in, so this may not work.
- teach the scheduler about the cost of migrating vcpus
The first approach will solve itself eventually, though slowly if the current slow rate of aio merging continues. We'll have to do the second.

Quoted 5 lines
Quoted 3 lines

if you prefer the first one, Sébastien will release very soon aio patches for 2.6.20 with an up-to-date libposix-aio.

Hi Laurent,

Hi Anthony,

Quoted 8 lines

I gave that a shot a little bit ago. Ran into two problems.

1) Couldn't avoid linking to -lrt as QEMU uses time functions from it.
2) While I could get things compiling (with patches), QEMU would SEGV almost immediately.

Could you guys maybe give compiling QEMU w/libposix-aio a shot? I'm really interested to see if it makes a difference.

Ok, I take a look at this.

Regards,
Laurent

Signature · 3 lines
------------- Laurent.Vivier-6ktuUTfB/[email protected]  --------------
       "Any sufficiently advanced technology is
  indistinguishable from magic." - Arthur C. Clarke

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

#36 lore raw
In reply to a message not in the archive
Quoted 51 lines · Anthony Liguori
Quoted 38 lines · Laurent Vivier
Quoted 27 lines · Avi Kivity
Quoted 13 lines · Aurelien Jarno
Quoted 5 lines

What is your disk image file format, or are you using a partition?

I am using a raw image file on an ext3 partition.

Quoted 4 lines

Do the results change (on kvm-14) if you pin the guest to a core with 'taskset 1 qemu ...'

Quoted 8 lines

Bingo. It now works even faster than kvm-13!

real    0m22.307s
user    0m13.935s
sys     0m4.720

I'm guessing this is due to the glibc aio implementation, which uses threads instead of true aio. The threads may cause the vcpu to migrate frequently from one code to another.

Quoted 9 lines

There are two possible solutions:

- use native aio from http://www.bullopensource.org/posix/. I think
the aio signal patches are not yet in, so this may not work.
- teach the scheduler about the cost of migrating vcpus
The first approach will solve itself eventually, though slowly if the current slow rate of aio merging continues. We'll have to do the second.

Quoted 3 lines

if you prefer the first one, Sébastien will release very soon aio patches for 2.6.20 with an up-to-date libposix-aio.

Hi Laurent,

I gave that a shot a little bit ago. Ran into two problems.

1) Couldn't avoid linking to -lrt as QEMU uses time functions from it.
2) While I could get things compiling (with patches), QEMU would SEGV almost immediately.

Quoted 12 lines

Could you guys maybe give compiling QEMU w/libposix-aio a shot? I'm really interested to see if it makes a difference.

Regards,

Anthony Liguori

Quoted 5 lines

[advertising] Keep an eye on the bullopensource website. [/advertising] :-P

Regards,
Laurent

OK, I didn' have time to test the performance of the result, but you can find attached some patches to enable libposix-aio with kvm-14.

first take last patches for linux-2.6.20 and libposix-aio-.0.8.2 from website :

http://sourceforge.net/projects/paiol

if you are using AMD64, you must patch libposix-aio because there is a little problem remaining to detect lio_submit syscall (first attachment)

then apply following patch to kvm-14 (second attachment).

It works fine on my system except when I use "-hda /dev/sdb" : qemu crashes just after mounting filesystems when "init" tries to set kernel parameters with "sysctl" (I use a debian 4.0). If I boot in emergency mode, mounting manually filesystems and running manually sysctl, all works fine. It looks like a synchronization problem.

There are remaining issues : libposix-aio uses kernel AIO, so files must be opened using O_DIRECT and buffers must be aligned. libposix-aio is able to manage other cases but this has a performance cost.

Aurélien, do you have any time to test this on your system ?

Regards,
Laurent

Signature · 3 lines
------------- Laurent.Vivier-6ktuUTfB/[email protected]  --------------
       "Any sufficiently advanced technology is
  indistinguishable from magic." - Arthur C. Clarke
configure.ac +21 -2
@@ -217,6 +217,24 @@
 
 # test if kernel supports signal on LIO completion
 
+case $host_cpu in
+
+        x86_64*)
+                lio_submit_syscall=280
+        ;;
+  
+        i386*)
+                lio_submit_syscall=320
+        ;;
+	*)
+		AC_MSG_WARN(unknown syscall number for lio_submit)
+		lio_submit_syscall=
+        ;;
+esac
+
+if test "${lio_submit_syscall}" != ""
+then
+
 AC_ARG_ENABLE(
 	[lio-signal],
 	[AS_HELP_STRING(--enable-lio-signal,
@@ -236,7 +254,7 @@
 
 #define IO_CMD_PWRITE 1
 
-#define SYS_lio_submit 320
+#define SYS_lio_submit ${lio_submit_syscall}
 
 #define LIO_WAIT   0
 #define LIO_NOWAIT 1
@@ -397,7 +415,7 @@
 
 #define IO_CMD_PWRITE 1
 
-#define SYS_lio_submit 320
+#define SYS_lio_submit ${lio_submit_syscall}
 
 #define LIO_WAIT   0
 #define LIO_NOWAIT 1
@@ -514,6 +532,7 @@
 			Define to 1 if kernel supports LIO wait)],[ac_lio_wait=no
 	AC_MSG_RESULT(no)
 ],[]))
+fi # end of test lio_submit_syscall
 
 # check for support of fd for io_cancel
 
configure +1 -1
Index: kvm-14/configure
===================================================================
@@ -87,7 +87,7 @@
     --enable-kvm --kernel-path="$libkvm_kerneldir" \
     --enable-alsa \
     ${disable_gcc_check:+"--disable-gcc-check"} \
-    --prefix="$prefix"
+    --prefix="$prefix" --enable-libposix-aio
 )
 
 
qemu/Makefile +4 -0
Index: kvm-14/qemu/Makefile
===================================================================
@@ -27,10 +27,14 @@
 ifndef CONFIG_DARWIN
 ifndef CONFIG_WIN32
 ifndef CONFIG_SOLARIS
+ifdef USE_LIBPOSIX_AIO
+LIBS+=-lposix-aio
+else
 LIBS+=-lrt
 endif
 endif
 endif
+endif
 
 all: $(TOOLS) $(DOCS) recurse-all
 
qemu/Makefile.target +5 -1
Index: kvm-14/qemu/Makefile.target
===================================================================
@@ -452,7 +452,11 @@
 ifndef CONFIG_DARWIN
 ifndef CONFIG_WIN32
 ifndef CONFIG_SOLARIS
-VL_LIBS=-lutil -lrt -luuid
+VL_LIBS=-lutil
+ifdef USE_LIBPOSIX_AIO
+VL_LIBS+=-lposix-aio
+endif
+VL_LIBS+=-lrt -luuid
 endif
 endif
 endif
qemu/block-raw.c +1 -1
Index: kvm-14/qemu/block-raw.c
===================================================================
@@ -197,7 +197,7 @@
     act.sa_handler = aio_signal_handler;
     sigaction(aio_sig_num, &act, NULL);
 
-#if defined(__GLIBC__) && defined(__linux__)
+#if defined(__GLIBC__) && defined(__linux__) && !defined(USE_LIBPOSIX_AIO)
     {
         /* XXX: aio thread exit seems to hang on RedHat 9 and this init
            seems to fix the problem. */
qemu/configure +7 -0
Index: kvm-14/qemu/configure
===================================================================
@@ -99,6 +99,7 @@
 linux_user="no"
 darwin_user="no"
 build_docs="no"
+libposix_aio="no"
 uname_release=""
 
 # OS specific
@@ -262,6 +263,8 @@
   ;;
   --enable-uname-release=*) uname_release="$optarg"
   ;;
+  --enable-libposix-aio) libposix_aio="yes"
+  ;;
   esac
 done
 
@@ -312,6 +315,7 @@
 echo "  --fmod-lib               path to FMOD library"
 echo "  --fmod-inc               path to FMOD includes"
 echo "  --enable-uname-release=R Return R for uname -r in usermode emulation"
+echo "  --enable-libposix-aio    use libposix-aio instead of glibc aio"
 echo ""
 echo "NOTE: The object files are built at the place where configure is launched"
 exit 1
@@ -635,6 +639,7 @@
 echo "kqemu support     $kqemu"
 echo "kvm support       $kvm"
 echo "Documentation     $build_docs"
+echo "libposix-aio      $libposix_aio"
 [ ! -z "$uname_release" ] && \
 echo "uname -r          $uname_release"
 
@@ -795,6 +800,8 @@
   echo "#define MAP_ANONYMOUS MAP_ANON" >> $config_h
   echo "#define _BSD 1" >> $config_h
 fi
+echo "USE_LIBPOSIX_AIO=yes" >> $config_mak
+echo "#define USE_LIBPOSIX_AIO 1" >> $config_h
 
 echo "#define CONFIG_UNAME_RELEASE \"$uname_release\"" >> $config_h
 

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

A message that is not in the archive (sent off-list or not yet received): 45DDC641.3030001-6ktuUTfB/[email protected]

author
#38 lore raw
Quoted 5 lines · Laurent Vivier

There are remaining issues : libposix-aio uses kernel AIO, so files must be opened using O_DIRECT and buffers must be aligned. libposix-aio is able to manage other cases but this has a performance cost.

Because the guest is also doing dma, the buffers are expected to be aligned (it might be different if using pio, but unlikely).

Signature · 8 lines
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): 45DDC6F3.8080104-atKUWr5tajBWk0Htik3J/[email protected]

#40 lore raw
Quoted 11 lines · Avi Kivity
Quoted 7 lines · Laurent Vivier

There are remaining issues : libposix-aio uses kernel AIO, so files must be
opened using O_DIRECT and buffers must be aligned. libposix-aio is able to
manage other cases but this has a performance cost.

Because the guest is also doing dma, the buffers are expected to be aligned (it might be different if using pio, but unlikely).

The IDE emulation always uses a temporary buffer. This is partially because you cannot always rely on being able to use an offset on phys_ram_base (when doing user only emulation).

We can pretty easily patch this code though to use the DMA address directly instead of the temporary buffer in the non PIO case. We still have to define a raw_pread() that is capable of emulating sector alignment though since Linux will use PIO at first.

Regards,

Anthony Liguori

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#42 lore raw
Quoted 10 lines · Anthony Liguori
Quoted 9 lines · Avi Kivity
Quoted 7 lines · Laurent Vivier

There are remaining issues : libposix-aio uses kernel AIO, so files must be
opened using O_DIRECT and buffers must be aligned. libposix-aio is able to
manage other cases but this has a performance cost.

Quoted 6 lines
Quoted 2 lines

Because the guest is also doing dma, the buffers are expected to be aligned (it might be different if using pio, but unlikely).

The IDE emulation always uses a temporary buffer. This is partially because you cannot always rely on being able to use an offset on phys_ram_base (when doing user only emulation).

Why is that?

Quoted 5 lines

We can pretty easily patch this code though to use the DMA address directly instead of the temporary buffer in the non PIO case. We still have to define a raw_pread() that is capable of emulating sector alignment though since Linux will use PIO at first.

Sure. It seems worthwhile for mainline qemu too.

Signature · 8 lines
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): 45DDD30D.4000809-atKUWr5tajBWk0Htik3J/[email protected]

#44 lore raw
Quoted 11 lines · Avi Kivity
Quoted 10 lines · Anthony Liguori
Quoted 9 lines · Avi Kivity
Quoted 7 lines · Laurent Vivier

There are remaining issues : libposix-aio uses kernel AIO, so files must be
opened using O_DIRECT and buffers must be aligned. libposix-aio is able to
manage other cases but this has a performance cost.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

Because the guest is also doing dma, the buffers are expected to be aligned (it might be different if using pio, but unlikely).

The IDE emulation always uses a temporary buffer. This is partially because you cannot always rely on being able to use an offset on phys_ram_base (when doing user only emulation).

Why is that?

Well, user won't do IDE emulation, but cpu_physical_memory_rw has to jump through more hoops for IDE.

What's more relevant though, is it's possible for the guest to setup DMA to device memory, in which case, you need to route the IO to callbacks. That's pretty unlikely though so I think we can stream line it so that we maintain correctness while also getting less copies.

Regards,

Anthony Liguori

Quoted 10 lines
Quoted 6 lines

We can pretty easily patch this code though to use the DMA address directly instead of the temporary buffer in the non PIO case. We still have to define a raw_pread() that is capable of emulating sector alignment though since Linux will use PIO at first.

Sure. It seems worthwhile for mainline qemu too.

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): 45DDDD01.7030802-r/[email protected]

#46 lore raw
Quoted 18 lines · Anthony Liguori
Quoted 10 lines · Avi Kivity
Quoted 9 lines · Anthony Liguori
Quoted 8 lines · Avi Kivity
Quoted 7 lines · Laurent Vivier

There are remaining issues : libposix-aio uses kernel AIO, so files must be
opened using O_DIRECT and buffers must be aligned. libposix-aio is able to
manage other cases but this has a performance cost.

Quoted 6 lines
Quoted 5 lines
Quoted 2 lines

Because the guest is also doing dma, the buffers are expected to be aligned (it might be different if using pio, but unlikely).

The IDE emulation always uses a temporary buffer. This is partially because you cannot always rely on being able to use an offset on phys_ram_base (when doing user only emulation).

Why is that?

Quoted 7 lines

Well, user won't do IDE emulation, but cpu_physical_memory_rw has to jump through more hoops for IDE.

What's more relevant though, is it's possible for the guest to setup DMA to device memory, in which case, you need to route the IO to callbacks. That's pretty unlikely though so I think we can stream line it so that we maintain correctness while also getting less copies.

I'm not sure I had understood correctly your explanation, but I tried to remove buffer from IDE device. I made a (pseudo-)benchmark with "dd"

Applied patch is here:

http://www.bullopensource.org/posix/qemu/ide-directio.diff

(pseudo-)Benchmark is here:

kvm-14

dd if=/dev/hda of=/dev/null count=100000
51200000 bytes transferred in 4.735706 seconds (10811482 bytes/sec) dd if=/dev/hda of=/dev/null count=1000000
512000000 bytes transferred in 25.218732 seconds (20302369 bytes/sec)

kvm-14 + libposix-aio

dd if=/dev/hda of=/dev/null count=100000
51200000 bytes transferred in 3.335737 seconds (15348932 bytes/sec) dd if=/dev/hda of=/dev/null count=1000000
512000000 bytes transferred in 19.242217 seconds (26608161 bytes/sec)

kvm-14 + libposix-aio + ide-directio

dd if=/dev/hda of=/dev/null count=100000
51200000 bytes transferred in 2.695296 seconds (18996058 bytes/sec) dd if=/dev/hda of=/dev/null count=1000000
512000000 bytes transferred in 14.992513 seconds (34150379 bytes/sec)

qemu-0.9.0

dd if=/dev/hda of=/dev/null count=100000
1200000 bytes transferred in 2.694546 seconds (19001345 bytes/sec) dd if=/dev/hda of=/dev/null count=1000000
512000000 bytes transferred in 25.524641 seconds (20059048 bytes/sec)

qemu-0.9.0 + libposix-aio

dd if=/dev/hda of=/dev/null count=100000
51200000 bytes transferred in 2.859584 seconds (17904702 bytes/sec) dd if=/dev/hda of=/dev/null count=1000000
512000000 bytes transferred in 25.045071 seconds (20443144 bytes/sec)

qemu-0.9.0 + libposix-aio + ide-directio

dd if=/dev/hda of=/dev/null count=100000
51200000 bytes transferred in 2.897503 seconds (17670387 bytes/sec) dd if=/dev/hda of=/dev/null count=1000000
512000000 bytes transferred in 25.900654 seconds (19767841 bytes/sec)

qemu-0.9.0 + kqemu

dd if=/dev/hda of=/dev/null count=100000
51200000 bytes transferred in 6.204987 seconds (8251427 bytes/sec) dd if=/dev/hda of=/dev/null count=1000000
512000000 bytes transferred in 65.149972 seconds (7858791 bytes/sec)

qemu-0.9.0 + kqemu + libposix-aio

dd if=/dev/hda of=/dev/null count=100000
51200000 bytes transferred in 6.075537 seconds (8427239 bytes/sec) dd if=/dev/hda of=/dev/null count=1000000
512000000 bytes transferred in 57.506392 seconds (8903358 bytes/sec)

qemu-0.9.0 + kqemu + libposix-aio + ide-directio

dd if=/dev/hda of=/dev/null count=100000
51200000 bytes transferred in 6.083927 seconds (8415617 bytes/sec) dd if=/dev/hda of=/dev/null count=1000000
512000000 bytes transferred in 57.602499 seconds (8888503 bytes/sec)

Regards,
Laurent

Signature · 3 lines
------------- Laurent.Vivier-6ktuUTfB/[email protected]  --------------
       "Any sufficiently advanced technology is
  indistinguishable from magic." - Arthur C. Clarke

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

#47 lore raw
In reply to a message not in the archive
Quoted 82 lines · Laurent Vivier
Quoted 50 lines · Anthony Liguori
Quoted 38 lines · Laurent Vivier
Quoted 27 lines · Avi Kivity
Quoted 13 lines · Aurelien Jarno
Quoted 5 lines

What is your disk image file format, or are you using a partition?

I am using a raw image file on an ext3 partition.

Quoted 4 lines

Do the results change (on kvm-14) if you pin the guest to a core with 'taskset 1 qemu ...'

Quoted 8 lines

Bingo. It now works even faster than kvm-13!

real    0m22.307s
user    0m13.935s
sys     0m4.720

I'm guessing this is due to the glibc aio implementation, which uses threads instead of true aio. The threads may cause the vcpu to migrate frequently from one code to another.

Quoted 9 lines

There are two possible solutions:

- use native aio from http://www.bullopensource.org/posix/. I think
the aio signal patches are not yet in, so this may not work.
- teach the scheduler about the cost of migrating vcpus
The first approach will solve itself eventually, though slowly if the current slow rate of aio merging continues. We'll have to do the second.

Quoted 3 lines

if you prefer the first one, Sébastien will release very soon aio patches for 2.6.20 with an up-to-date libposix-aio.

Hi Laurent,

I gave that a shot a little bit ago. Ran into two problems.

1) Couldn't avoid linking to -lrt as QEMU uses time functions from it.
2) While I could get things compiling (with patches), QEMU would SEGV almost immediately.

Quoted 12 lines

Could you guys maybe give compiling QEMU w/libposix-aio a shot? I'm really interested to see if it makes a difference.

Regards,

Anthony Liguori

Quoted 5 lines

[advertising] Keep an eye on the bullopensource website. [/advertising] :-P

Regards,
Laurent

OK, I didn' have time to test the performance of the result, but you can find attached some patches to enable libposix-aio with kvm-14.

first take last patches for linux-2.6.20 and libposix-aio-.0.8.2 from website :

http://sourceforge.net/projects/paiol

if you are using AMD64, you must patch libposix-aio because there is a little problem remaining to detect lio_submit syscall (first attachment)

then apply following patch to kvm-14 (second attachment).

It works fine on my system except when I use "-hda /dev/sdb" : qemu crashes just after mounting filesystems when "init" tries to set kernel parameters with "sysctl" (I use a debian 4.0). If I boot in emergency mode, mounting manually filesystems and running manually sysctl, all works fine. It looks like a synchronization problem.

Quoted 5 lines

There are remaining issues : libposix-aio uses kernel AIO, so files must be opened using O_DIRECT and buffers must be aligned. libposix-aio is able to manage other cases but this has a performance cost.

Aurélien, do you have any time to test this on your system ?

It took me some time to test it, I was not at home and I prefer to be near the machine when I change the kernel!

I have just tried your patches, and I can say it works. I get the same times (minor measurement issues) as when using kvm without your patches and with taskset. Thanks a lot!

Would it be possible to merge the kvm part, and maybe to send it to the qemu mailing list?

Regards,
Aurelien

Signature · 10 lines
  .''`.  Aurelien Jarno	            | GPG: 1024D/F1BCDB73
 : :' :  Debian developer           | Electrical Engineer
 `. `'   [email protected]         | [email protected]
   `-    people.debian.org/~aurel32 | www.aurel32.net

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): [email protected]

#49 lore raw
Quoted 82 lines · Aurelien Jarno
Quoted 81 lines · Laurent Vivier
Quoted 50 lines · Anthony Liguori
Quoted 38 lines · Laurent Vivier
Quoted 27 lines · Avi Kivity
Quoted 13 lines · Aurelien Jarno
Quoted 5 lines

What is your disk image file format, or are you using a partition?

I am using a raw image file on an ext3 partition.

Quoted 4 lines

Do the results change (on kvm-14) if you pin the guest to a core with 'taskset 1 qemu ...'

Quoted 8 lines

Bingo. It now works even faster than kvm-13!

real    0m22.307s
user    0m13.935s
sys     0m4.720

I'm guessing this is due to the glibc aio implementation, which uses threads instead of true aio. The threads may cause the vcpu to migrate frequently from one code to another.

Quoted 9 lines

There are two possible solutions:

- use native aio from http://www.bullopensource.org/posix/. I think
the aio signal patches are not yet in, so this may not work.
- teach the scheduler about the cost of migrating vcpus
The first approach will solve itself eventually, though slowly if the current slow rate of aio merging continues. We'll have to do the second.

Quoted 3 lines

if you prefer the first one, Sébastien will release very soon aio patches for 2.6.20 with an up-to-date libposix-aio.

Hi Laurent,

I gave that a shot a little bit ago. Ran into two problems.

1) Couldn't avoid linking to -lrt as QEMU uses time functions from it.
2) While I could get things compiling (with patches), QEMU would SEGV almost immediately.

Quoted 12 lines

Could you guys maybe give compiling QEMU w/libposix-aio a shot? I'm really interested to see if it makes a difference.

Regards,

Anthony Liguori

Quoted 5 lines

[advertising] Keep an eye on the bullopensource website. [/advertising] :-P

Regards,
Laurent

OK, I didn' have time to test the performance of the result, but you can find attached some patches to enable libposix-aio with kvm-14.

first take last patches for linux-2.6.20 and libposix-aio-.0.8.2 from website :

http://sourceforge.net/projects/paiol

if you are using AMD64, you must patch libposix-aio because there is a little problem remaining to detect lio_submit syscall (first attachment)

then apply following patch to kvm-14 (second attachment).

It works fine on my system except when I use "-hda /dev/sdb" : qemu crashes just after mounting filesystems when "init" tries to set kernel parameters with "sysctl" (I use a debian 4.0). If I boot in emergency mode, mounting manually filesystems and running manually sysctl, all works fine. It looks like a synchronization problem.

Quoted 8 lines
Quoted 5 lines

There are remaining issues : libposix-aio uses kernel AIO, so files must be opened using O_DIRECT and buffers must be aligned. libposix-aio is able to manage other cases but this has a performance cost.

Aurélien, do you have any time to test this on your system ?

It took me some time to test it, I was not at home and I prefer to be near the machine when I change the kernel!

thank you very much to have tested it : I was not able to reproduce your problem with debian/linux guest on my intel xeon MP.

Quoted 3 lines

I have just tried your patches, and I can say it works. I get the same times (minor measurement issues) as when using kvm without your patches and with taskset. Thanks a lot!

Good news.

Quoted 2 lines

Would it be possible to merge the kvm part, and maybe to send it to the qemu mailing list?

I can try to send it to qemu mailing list but I think it is not useful for pure qemu. It seems to improve performance only with kvm.

Avi, your opinion ?

Quoted 2 lines

Regards,
Aurelien

Regards,
Laurent

Signature · 3 lines
------------- Laurent.Vivier-6ktuUTfB/[email protected]  --------------
       "Any sufficiently advanced technology is
  indistinguishable from magic." - Arthur C. Clarke

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel

A message that is not in the archive (sent off-list or not yet received): 45E30354.3020407-6ktuUTfB/[email protected]

author
#51 lore raw
Quoted 9 lines · Laurent Vivier
Quoted 4 lines

Would it be possible to merge the kvm part, and maybe to send it to the qemu mailing list?

I can try to send it to qemu mailing list but I think it is not useful for pure qemu. It seems to improve performance only with kvm.

Avi, your opinion ?

As it doesn't involve code changes, I see no reason why it can't be merged into qemu. The case would probably be stronger if distributions carried libposix-aio -- do they?

What are the long term plans for libposix-aio? Do you plan to contribute it into glibc?

Also, it probably relies on kernel patches to get signal notification to work. When are these patches going to be merged?

Signature · 8 lines
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

A message that is not in the archive (sent off-list or not yet received): 45E3097C.7020606-atKUWr5tajBWk0Htik3J/[email protected]

#53 lore raw
Quoted 16 lines · Avi Kivity
Quoted 10 lines · Laurent Vivier
Quoted 4 lines

Would it be possible to merge the kvm part, and maybe to send it to the qemu mailing list?

I can try to send it to qemu mailing list but I think it is not useful for pure
qemu. It seems to improve performance only with kvm.

Avi, your opinion ?

As it doesn't involve code changes, I see no reason why it can't be merged into qemu. The case would probably be stronger if distributions carried libposix-aio -- do they?

Quoted 5 lines

What are the long term plans for libposix-aio? Do you plan to contribute it into glibc?

Also, it probably relies on kernel patches to get signal notification to work. When are these patches going to be merged?

AIO completion signal has been added to the -mm tree, but support in glibc is not ready. Ulrich Drepper should work on this, but I think Sébastien already did this work.

So, libposix-aio should be considered only as experimental until its integration in glibc.

Sébastien Dugué is the maintainer and main contributor of libposix-aio, he could correct me if I'm wrong.

Regards,
Laurent

Signature · 3 lines
------------- Laurent.Vivier-6ktuUTfB/[email protected]  --------------
       "Any sufficiently advanced technology is
  indistinguishable from magic." - Arthur C. Clarke

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV

_______________________________________________
kvm-devel mailing list [email protected]
https://lists.sourceforge.net/lists/listinfo/kvm-devel