Enhance perf to support KVM

Quoted 18 lines · Yanmin

Ingo,

I did some testing with KVM virtualization. perf shows vmx_vcpu_run consumes more than 50% cpu time. Actually, the info is incorrect because when perf counter overflows and NMI is triggered, vm exit to function vmx_vcpu_run, then vmx_vcpu_run triggers a software NMI so perf event is notified. perf just checks regs which just saves the address of vmx_vcpu_run.

I want to enhance perf to collect real guest os address.

Below is the design.
KVM uses multi-thread model. Every guest os is a process of multi-thread.

1) Kernel:
Add a per_cpu var and some functions, so KVM records interrupted
guest os address before triggering the software NMI. perf event would check the per_cpu var to use it if it's not zero, or just goes though the old path.

Quoted 5 lines

2) User space: Add a new parameter to perf-top and perf-report, such like -g pid:guest_os_vmlinux_path. Command perf parses the guest os kernel image to collect symbols. Change perf to summarize results based on pid. Another direction is to use the new parameter -g only when old parameter -p is defined. Perf just needs separate native kernel and guest os kernel.

-g is already taken :-)

One thing I worry about is making sense of the guest data, it might be possible to sorta make sense of the main kernel image, but after that its going to be 'interesting' in deed.

You're going to have to extend PERF_RECORD_MISC_* though, perhaps you can reuse CPUMODE_UNKNOWN for GUEST.

The callchain stuff already has GUEST context identifiers, however determining KERNEL/USER context might be hard and interpreting it is going to be harder still since we don't have map information for the guest.

2) We couldn't get guest os kernel/user stack data in an easy way, so we might not support callchain feature of tool perf. A work around is KVM copies kernel stack data out, so we could at least support guest os kernel callchain.

Quoted 5 lines · Zhang, Yanmin

2) We couldn't get guest os kernel/user stack data in an easy way, so we might not support callchain feature of tool perf. A work around is KVM copies kernel stack data out, so we could at least support guest os kernel callchain.

If the guest is Linux, KVM can get all the info we need.

While the PMU event itself might trigger in an NMI (where we cannot access most of KVM's data structures safely), for this specific case of KVM instrumentation we can delay the processing to a more appropriate time - in fact we can do it in the KVM thread itself.

We can do that because we just triggered a VM exit, so the VM state is for all purposes frozen (as far as this virtual CPU goes).

Which egives us plenty of time and opportunity to piggy back to the KVM thread, look up the guest stack, process/fill the MMU cache as we walk the guest page tables, etc. etc.

It would need some minimal callback facility towards KVM, triggered by a perf event PMI.

One additional step needed is to get symbol information from the guest, and to integrate it into the symbol cache on the host side in ~/.debug. We already support cross-arch symbols and 'perf archive', so the basic facilities are there for that. So you can profile on 32-bit PA-RISC and type 'perf report' on 64-bit x86 and get all the right info.

For this to work across a guest, a gateway is needed towards the guest. There's several ways to achieve this. The most practical would be two steps:

- a user-space facility to access guest images/libraries. (say via ssh, or just a plain TCP port) This would be useful for general 'remote profiling' sessions as well, so it's not KVM specific - it would be useful for remote debugging.

- The guest /proc/kallsyms (and vmlinux) could be accessed via that channel as well.

(Note that this is purely for guest symbol space access - all the profiling data itself comes via the host kernel.)

In theory we could build some sort of 'symbol server' facility into the kernel, which could be enabled in guest kernels too - but i suspect existing, user-space transports go most of the way already.

Quoted 6 lines · Ingo Molnar

Quoted 4 lines · Zhang, Yanmin

2) We couldn't get guest os kernel/user stack data in an easy way, so we might not support callchain feature of tool perf. A work around is KVM copies kernel stack data out, so we could at least support guest os kernel callchain.

Quoted 6 lines

If the guest is Linux, KVM can get all the info we need.

While the PMU event itself might trigger in an NMI (where we cannot access most of KVM's data structures safely), for this specific case of KVM instrumentation we can delay the processing to a more appropriate time - in fact we can do it in the KVM thread itself.

The nmi will be a synchronous event: it happens in guest context, and we program the hardware to intercept nmis, so we just get an exit telling us that an nmi has happened.

(would also be interesting to allow the guest to process the nmi directly in some scenarios, though that would require that there be no nmi sources on the host).

Quoted 2 lines

We can do that because we just triggered a VM exit, so the VM state is for all purposes frozen (as far as this virtual CPU goes).

Yes.

Quoted 4 lines

Which egives us plenty of time and opportunity to piggy back to the KVM thread, look up the guest stack, process/fill the MMU cache as we walk the guest page tables, etc. etc.

Quoted 2 lines

It would need some minimal callback facility towards KVM, triggered by a perf event PMI.

Since the event is synchronous and kvm is aware of it we don't need a callback; kvm can call directly into perf with all the information.

Quoted 20 lines

One additional step needed is to get symbol information from the guest, and to integrate it into the symbol cache on the host side in ~/.debug. We already support cross-arch symbols and 'perf archive', so the basic facilities are there for that. So you can profile on 32-bit PA-RISC and type 'perf report' on 64-bit x86 and get all the right info.

For this to work across a guest, a gateway is needed towards the guest. There's several ways to achieve this. The most practical would be two steps:

- a user-space facility to access guest images/libraries. (say via ssh, or just a plain TCP port) This would be useful for general 'remote profiling' sessions as well, so it's not KVM specific - it would be useful for remote debugging.

- The guest /proc/kallsyms (and vmlinux) could be accessed via that channel as well.

(Note that this is purely for guest symbol space access - all the profiling data itself comes via the host kernel.)

Quoted 3 lines

In theory we could build some sort of 'symbol server' facility into the kernel, which could be enabled in guest kernels too - but i suspect existing, user-space transports go most of the way already.

There is also vmchannel aka virtio-serial, a guest-to-host communication channel.

Quoted 25 lines

Quoted 25 lines

One additional step needed is to get symbol information from the guest, and to integrate it into the symbol cache on the host side in ~/.debug. We already support cross-arch symbols and 'perf archive', so the basic facilities are there for that. So you can profile on 32-bit PA-RISC and type 'perf report' on 64-bit x86 and get all the right info.

For this to work across a guest, a gateway is needed towards the guest. There's several ways to achieve this. The most practical would be two steps:

- a user-space facility to access guest images/libraries. (say via ssh, or just a plain TCP port) This would be useful for general 'remote profiling' sessions as well, so it's not KVM specific - it would be useful for remote debugging.

- The guest /proc/kallsyms (and vmlinux) could be accessed via that channel as well.

(Note that this is purely for guest symbol space access - all the profiling data itself comes via the host kernel.)

In theory we could build some sort of 'symbol server' facility into the kernel, which could be enabled in guest kernels too - but i suspect existing, user-space transports go most of the way already.

Quoted 3 lines

There is also vmchannel aka virtio-serial, a guest-to-host communication channel.

Basically what is needed is plain filesystem access - properly privileged. So doing this via a vmchannel would be nice, but for the symbol extraction it would be a glorified NFS server in essence.

Do you have (or plan) any turn-key 'access to all files of the guest' kind of guest-transparent facility that could be used for such purposes?

That would
have various advantages over a traditional explicit file server approach:

- it would not contaminate the guest port space

- no guest side configuration needed (the various oprofile remote daemons always sucked as they needed extra setup)

- it might even be used with a guest that does no networking

- if done fully in the kernel it could be done with a fully 'unaware' guest, etc.

Quoted 2 lines

Do you have (or plan) any turn-key 'access to all files of the guest' kind of guest-transparent facility that could be used for such purposes?

Not really. The guest and host admins are usually different people, who may, being admins, even actively hate each other. The guest admin would probably regard it as a security hole. It's probably useful for the single-host scenario, and of course for developers.

Quoted 8 lines · Avi Kivity

Quoted 3 lines

Do you have (or plan) any turn-key 'access to all files of the guest' kind of guest-transparent facility that could be used for such purposes?

Not really. The guest and host admins are usually different people, who may, being admins, even actively hate each other. The guest admin would probably regard it as a security hole. It's probably useful for the single-host scenario, and of course for developers.

Sounds like an exceedingly silly argument to me - the host admin is the king in any case.

Your argument boils down to: 'dont offer transparent, turn-key solutions because some might object to the functionality they offer for all the wrong reasons'. Which does not withstand elementary scrutiny.

This is a basic usability issue, and affects many parts of the KVM universe.

Really, it's by far the most fubar-ed notion of KVM. You are pushing _way_ too much to user-space into different modules and maintenance domains, and user-space forks those bits, fragments, diverts, delays and messes up basic features in the usual fashion.

The result is a basic out-of-box virtualization experience that sucks even these days.

Nobody is really 'in charge' of how KVM gets delivered to the user. You isolated the fun kernel part for you and pushed out the boring bits to user-space. So if mundane things like mouse integration sucks 'hey that's a user-space tooling problem', if file integration sucks then 'hey, that's an admin problem', if it cannot be used over the network 'hey, that's an Xorg problem', etc. etc.

You basically have given up control over the quality of KVM by pushing so many aspects of it to user-space and letting it rot there.

Sure the design looks somewhat cleaner on paper, but if the end result is not helped by it then over-modularization sure can hurt ...

( Note that i dont mind user-space tooling per se, as long as it sits together
with the kernel bits and gets developed, packaged and given to the user in the same domain. )

And that's a key conceptual area were tools/perf/ differs: it's an integrated, turn-key solution that you can really rely on. We take responsibility for the full thing, no ifs and when. And if you cannot rely on your instrumentation tooling as a single unit you cannot use it, simple as that. (that is a key mistake Oprofile made a decade ago too btw.)

So i can see some upcoming culture friction with standing KVM principles there ;-)

Not really. The guest and host admins are usually different people, who may, being admins, even actively hate each other. The guest admin would probably regard it as a security hole. It's probably useful for the single-host scenario, and of course for developers.

Quoted 5 lines · Avi Kivity

Not really. The guest and host admins are usually different people, who may, being admins, even actively hate each other. The guest admin would probably regard it as a security hole. It's probably useful for the single-host scenario, and of course for developers.

LOL, let me be the malicious host admin, then you can be the guest, there is no way you can protect yourself. If you don't trust the host, don't use it.

All your IO flows through the host, all your sekrit keys are in memory, there is no security.

Quoted 2 lines

You basically have given up control over the quality of KVM by pushing so many aspects of it to user-space and letting it rot there.

That's wrong on so many levels. First, nothing is rotting in userspace, qemu is evolving faster than kvm is. If I pushed it into the kernel then development pace would be much slower (since kernel development is harder), quality would be lower (less infrastructure, any bug is a host crash or security issue), and I personally would be totally swamped.

Quoted 6 lines

[...] You are pushing _way_ too much to user-space into different modules and maintenance domains, [...]

( Note that i dont mind user-space tooling per se, as long as it sits together
with the kernel bits and gets developed, packaged and given to the user in the same domain. ) [...]

Quoted 2 lines

Sure the design looks somewhat cleaner on paper, but if the end result is not helped by it then over-modularization sure can hurt ...

Run 'rpm -qa' one of these days. Modern software is modular, that's the only way to manage it.

Quoted 3 lines · Avi Kivity

Quoted 3 lines

You basically have given up control over the quality of KVM by pushing so many aspects of it to user-space and letting it rot there.

Quoted 6 lines

That's wrong on so many levels. First, nothing is rotting in userspace, qemu is evolving faster than kvm is. If I pushed it into the kernel then development pace would be much slower (since kernel development is harder), quality would be lower (less infrastructure, any bug is a host crash or security issue), and I personally would be totally swamped.

That was not what i suggested tho. tools/kvm/ would work plenty fine.

Quoted 7 lines · As i

Quoted 7 lines

[...] You are pushing _way_ too much to user-space into different modules and maintenance domains, [...]

( Note that i dont mind user-space tooling per se, as long as it sits together
with the kernel bits and gets developed, packaged and given to the user in the same domain. ) [...]

Quoted 6 lines

Quoted 3 lines

Sure the design looks somewhat cleaner on paper, but if the end result is not helped by it then over-modularization sure can hurt ...

Run 'rpm -qa' one of these days. Modern software is modular, that's the only way to manage it.

Of course rpm -qa shows cases where modularization works. But my point was over-modularization, which due to the KVM/qemu split we all suffer from.

Modularizing along the wrong interface is worse than not modularizing something that could be. So when designing software you generally want to err on the side of _under_-modularizing. It's always very easy to split stuff up, when there's a really strong technical argument for it. It's very hard to pull the broken pieces back together though once they are in difference domains of maintanence - as then it's usually social integration that has to happen, which is always harder than a technical split-up.

Quoted 1 line

That was not what i suggested tho. tools/kvm/ would work plenty fine.

I'll wait until we have tools/libc and tools/X. After all, they affect a lot more people and are concerned with a lot more kernel/user interfaces than kvm.

Quoted 5 lines · Avi Kivity

Quoted 1 line

That was not what i suggested tho. tools/kvm/ would work plenty fine.

I'll wait until we have tools/libc and tools/X. After all, they affect a lot more people and are concerned with a lot more kernel/user interfaces than kvm.

So your answer can be summed up as: 'we wont do what makes sense technically because others suck even more' ?

Quoted 6 lines · Avi Kivity

Quoted 2 lines

That was not what i suggested tho. tools/kvm/ would work plenty fine.

I'll wait until we have tools/libc and tools/X. After all, they affect a lot more people and are concerned with a lot more kernel/user interfaces than kvm.

So your answer can be summed up as: 'we wont do what makes sense technically because others suck even more' ?

And it's not just the kernel<->user interface (which btw., for the case of X is far narrower than what KVM currently has to Qemu).

The issue is a basic question of software design: does kvm-qemu really make as much sense without the kernel component as with it? The answer is: it will borderline-work with CPU emulation (and i'm sure there are people making use of it that way), but 90%+ of the userbase uses it with KVM and vice versa. It is really a single logical component as far as maintenance goes, and tools/kvm/ would make quite a bit of sense.

Quoted 8 lines · Ingo Molnar

Quoted 5 lines · Avi Kivity

Quoted 2 lines

That was not what i suggested tho. tools/kvm/ would work plenty fine.

I'll wait until we have tools/libc and tools/X. After all, they affect a lot more people and are concerned with a lot more kernel/user interfaces than kvm.

Quoted 2 lines

So your answer can be summed up as: 'we wont do what makes sense technically because others suck even more' ?

I can sum up your this remark as 'whenever you disagree with me, I will rephrase your words to make you look like an idiot'.

Quoted 15 lines · Avi Kivity

Quoted 13 lines · Ingo Molnar

Quoted 7 lines · Avi Kivity

Quoted 3 lines

That was not what i suggested tho. tools/kvm/ would work plenty fine.

I'll wait until we have tools/libc and tools/X. After all, they affect a lot more people and are concerned with a lot more kernel/user interfaces than kvm.

So your answer can be summed up as: 'we wont do what makes sense technically because others suck even more' ?

Quoted 3 lines

I can sum up your this remark as 'whenever you disagree with me, I will rephrase your words to make you look like an idiot'.

Two points:

1)

You can try to ridicule me if you want,

but do you actually claim that my
summary is inaccurate?

I do claim it's a substantially accurate summary: you said you will (quote:) "wait with tools/kvm/ until we have tools/libc and tools/X".

I do think tools/X and tools/libc would make quite a bit of sense - this is one of the better design aspects of FreeBSD et al. It's a mistake that it's not being done.

2)

I used a question mark (the sentence was not a statement of fact), and you have no obligation to agree with the summary i provided.

Nobody is really 'in charge' of how KVM gets delivered to the user. You isolated the fun kernel part for you and pushed out the boring bits to user-space. So if mundane things like mouse integration sucks 'hey that's a user-space tooling problem', if file integration sucks then 'hey, that's an admin problem', if it cannot be used over the network 'hey, that's an Xorg problem', etc. etc.

Basically what is needed is plain filesystem access - properly privileged. So doing this via a vmchannel would be nice, but for the symbol extraction it would be a glorified NFS server in essence.

Do you have (or plan) any turn-key 'access to all files of the guest' kind of guest-transparent facility that could be used for such purposes? That would have various advantages over a traditional explicit file server approach:

- it would not contaminate the guest port space

- no guest side configuration needed (the various oprofile remote daemons always sucked as they needed extra setup)

- it might even be used with a guest that does no networking

Quoted 6 lines · Ingo Molnar

Nobody is really 'in charge' of how KVM gets delivered to the user. You isolated the fun kernel part for you and pushed out the boring bits to user-space. So if mundane things like mouse integration sucks 'hey that's a user-space tooling problem', if file integration sucks then 'hey, that's an admin problem', if it cannot be used over the network 'hey, that's an Xorg problem', etc. etc.

btw, mouse integration works with -usbdevice tablet and recent Fedoras, 'it was an X.org driver problem'.

Really, I don't understand your problems.

Quoted 11 lines · Avi Kivity

Quoted 6 lines · Ingo Molnar

Nobody is really 'in charge' of how KVM gets delivered to the user. You isolated the fun kernel part for you and pushed out the boring bits to user-space. So if mundane things like mouse integration sucks 'hey that's a user-space tooling problem', if file integration sucks then 'hey, that's an admin problem', if it cannot be used over the network 'hey, that's an Xorg problem', etc. etc.

btw, mouse integration works with -usbdevice tablet and recent Fedoras, 'it was an X.org driver problem'.

Really, I don't understand your problems.

I run bleeding edge rawhide on my main desktop so i just tried latest & greatest KVM and Qemu bits and started up kvm-qemu with some Fedora and XP images i had around:

2.6.33-0.44.rc8.git0.fc13.x86_64
qemu-system-x86-0.12.2-6.fc13.x86_64

Here's my experience with it:

- qemu-kvm starts up with a miniature resolution by default. 640x480 - on my 1680x1050 laptop screen. It's so small that initially i even overlooked that i started it. It should multiplex pixels up to a reasonable screen size by default.

- The mouse is trapped straight away by default if you click into it. That's very annoying if you actually try to integrate a guest OS into your desktop
- it's not just 'another, slightly weird app' but a sticky, opinionated GUI component that you have to fight with all the time.

- Once trapped it's not obvious how to untrap the mouse. The qemu window title says:

QEMU: Press Ctrl-ALT to exit grab

Of course once you _know_ what a 'grab' is, you'll know where to look. At minimum it should say:

QEMU: Press Ctrl-ALT to exit mouse grab

But to first-time users it's an annoying trap of the mouse and with no obvious place to look for help. [besides, it doesnt tell which Ctrl and which ALT to use - it's the left side. The right side Ctrl does not work.]

- Graphics performance is awful even with the 640x480 miniature version. During bootup I can see it drawing single characters. This is a Core2 2.8GHz.

- Sound does not work by default. I have to go dig into command-line options to see that i need to add: "-soundhw all". Why isnt sound enabled by default?

- Qemu images are not integrated into the rest of the desktop. If i click on a Qemu image it says:

Could not display "/home/mingo/qemu/hda.img".

The file is of an unknown type.

10 years of Qemu and its base image format is still 'unknown' ?

- Random bugs. I tried to boot some old Fedora image i had around, it says:

spirit:~/qemu> qemu-kvm ./hda-fedora.img
kvm: unhandled exit 80000021
kvm_run returned -22

Bugs happen, but more important is what a user can do with it. To a user, what does this tell? Is it actionable? Does it give any URL to check? Any bug submit mechanism to follow? Does it even tell what the code itself thinks that happened? Nope - it just prints that error message (on the console, so to anyone starting it via a clicky interface wouldnt notice that there's something wrong) - and the guest session hanging indefinitely.

- When it boots up, the Qemu window flips around its size crazily, as the BIOS, the bootloader and the OS sets different screen resolutions. To the user that technical detail is immaterial, what matters is an amateurish-looking app that flips its window size as if it was an adware popup window trying to avoid being caught.

- There's no obvious way to activate paravirt drivers on the Windows side. There's no friendly "install guest drivers" button to click on with Qemu.

_Of course_ you will end up emulating hardware in KVM (and passing it through to the guest once it's clear that emulation performance sucks) and sooner or later you will end up requesting unreasonable things of the host kernel to achieve that ...

- Another small detail: there is zero on-screen help (beyond the Ctrl-ALT line) for a newbie to quickly find his way around it. No wiki address, no help, no nothing. There's not even any hint about what this window does. Which guest is it? In what state is that guest?

- But i'm a more advanced user so i dont need help screens, i knew that the "go full screen" hotkey is:

LeftCtrl-LeftALT-F

... except that it is a one-way road: pressing it for a second time does not restore the window, trapping me in the guest altogether! Ctrl-ALT does not exit the trap either. I had to shut down the guest to get back my X desktop.

etc., etc.

( I could go on and on about finer details of good integration, like the
difficulty of integrating host/guest files, networking, no way to quickly attach ISOs via that window, no way to activate networking, sound and no way to snapshot, no way to increase memory size except a command line option. )

etc - but that's not the point really: i only spent 10 minutes on this and i didnt try hard at all - _11_ bugs/annoyances from all across the functionality spectrum.

And the thing is _me_ reporting bugs does not matter at all in this picture, so please dont come with "why didnt you report this?".

_Anyone_ with half a brain who takes a critical look at this virtualization solution would notice the same. Still, it's essentially unchanged from 5 years ago.

Why is that so? I have outlined my opinion that this is due to the artificial package separation / over-modularization and no-one really being in charge of "KVM quality as a whole" - and i'm wondering what your theory is how such a state of affairs became possible.

I'm not trolling you at all: is it _really_ not obvious to you that the KVM/qemu usability status quo honestly sucks, to an unbiased observer?

And AFAICS KVM developers keep concentrating on all the wrong things due to that bad split/packaging: writing newer and newer low level kernel patches and optimizations which are nice but in large part irrelevant because the _fundamental basics_ of usability suck so much ... But to you it's probably just another external package so not really something you can do much about, right?

Really, the KVM design is so nice in many regards and Qemu has come forward leaps and bounds in the past few years as well, how can you miss such basic areas of weakness? 'First impression' is the thing that gives you new developers - it's any OSS project's bread and butter.

Quoted 9 lines · Avi Kivity

Quoted 7 lines · Ingo Molnar

Nobody is really 'in charge' of how KVM gets delivered to the user. You isolated the fun kernel part for you and pushed out the boring bits to user-space. So if mundane things like mouse integration sucks 'hey that's a user-space tooling problem', if file integration sucks then 'hey, that's an admin problem', if it cannot be used over the network 'hey, that's an Xorg problem', etc. etc.

Quoted 5 lines

btw, mouse integration works with -usbdevice tablet and recent Fedoras, 'it was an X.org driver problem'.

Really, I don't understand your problems.

I run bleeding edge rawhide on my main desktop so i just tried latest& greatest KVM and Qemu bits and started up kvm-qemu with some Fedora and XP images i had around:

   2.6.33-0.44.rc8.git0.fc13.x86_64
   qemu-system-x86-0.12.2-6.fc13.x86_64

Here's my experience with it:

- qemu-kvm starts up with a miniature resolution by default. 640x480 - on my 1680x1050 laptop screen. It's so small that initially i even overlooked that i started it. It should multiplex pixels up to a reasonable screen size by default.

- The mouse is trapped straight away by default if you click into it. That's very annoying if you actually try to integrate a guest OS into your desktop
- it's not just 'another, slightly weird app' but a sticky, opinionated GUI component that you have to fight with all the time.

- Once trapped it's not obvious how to untrap the mouse. The qemu window title says:

QEMU: Press Ctrl-ALT to exit grab

Of course once you _know_ what a 'grab' is, you'll know where to look. At minimum it should say:

QEMU: Press Ctrl-ALT to exit mouse grab

But to first-time users it's an annoying trap of the mouse and with no obvious place to look for help. [besides, it doesnt tell which Ctrl and which ALT to use - it's the left side. The right side Ctrl does not work.]

- Graphics performance is awful even with the 640x480 miniature version. During bootup I can see it drawing single characters. This is a Core2 2.8GHz.

- Sound does not work by default. I have to go dig into command-line options to see that i need to add: "-soundhw all". Why isnt sound enabled by default?

- Qemu images are not integrated into the rest of the desktop. If i click on a Qemu image it says:

Could not display "/home/mingo/qemu/hda.img".

The file is of an unknown type.

10 years of Qemu and its base image format is still 'unknown' ?

- Random bugs. I tried to boot some old Fedora image i had around, it says:

spirit:~/qemu> qemu-kvm ./hda-fedora.img
kvm: unhandled exit 80000021
kvm_run returned -22

Bugs happen, but more important is what a user can do with it. To a user, what does this tell? Is it actionable? Does it give any URL to check? Any bug submit mechanism to follow? Does it even tell what the code itself thinks that happened? Nope - it just prints that error message (on the console, so to anyone starting it via a clicky interface wouldnt notice that there's something wrong) - and the guest session hanging indefinitely.

- When it boots up, the Qemu window flips around its size crazily, as the BIOS, the bootloader and the OS sets different screen resolutions. To the user that technical detail is immaterial, what matters is an amateurish-looking app that flips its window size as if it was an adware popup window trying to avoid being caught.

- There's no obvious way to activate paravirt drivers on the Windows side. There's no friendly "install guest drivers" button to click on with Qemu.

_Of course_ you will end up emulating hardware in KVM (and passing it through to the guest once it's clear that emulation performance sucks) and sooner or later you will end up requesting unreasonable things of the host kernel to achieve that ...

- Another small detail: there is zero on-screen help (beyond the Ctrl-ALT line) for a newbie to quickly find his way around it. No wiki address, no help, no nothing. There's not even any hint about what this window does. Which guest is it? In what state is that guest?

- But i'm a more advanced user so i dont need help screens, i knew that the "go full screen" hotkey is:

LeftCtrl-LeftALT-F

... except that it is a one-way road: pressing it for a second time does not restore the window, trapping me in the guest altogether! Ctrl-ALT does not exit the trap either. I had to shut down the guest to get back my X desktop.

etc., etc.

( I could go on and on about finer details of good integration, like the
difficulty of integrating host/guest files, networking, no way to quickly attach ISOs via that window, no way to activate networking, sound and no way to snapshot, no way to increase memory size except a command line option. )

etc - but that's not the point really: i only spent 10 minutes on this and i didnt try hard at all - _11_ bugs/annoyances from all across the functionality spectrum.

And the thing is _me_ reporting bugs does not matter at all in this picture, so please dont come with "why didnt you report this?".

_Anyone_ with half a brain who takes a critical look at this virtualization solution would notice the same. Still, it's essentially unchanged from 5 years ago.

Why is that so? I have outlined my opinion that this is due to the artificial package separation / over-modularization and no-one really being in charge of "KVM quality as a whole" - and i'm wondering what your theory is how such a state of affairs became possible.

I'm not trolling you at all: is it _really_ not obvious to you that the KVM/qemu usability status quo honestly sucks, to an unbiased observer?

Second, it's not over-modularized. The modules are the individual components of the architecture. How would you propose to put it differently. They really can't naturally combine. And with the code quality of qemu in general being problematic by Linux kernel standards, it's not natural to move the device emulation directly into the kernel module. So this is why we are where we are today.

Third, it's the maintainers upstream who are in charge of KVM quality as a whole - when you are talking about upstream code quality, and the package maintainers who are in charge of KVM quality as a whole - when you are talking about a distro. This is nothing new - it's just a statement of decentralization.

Quoted 6 lines

Here's my experience with it:

- qemu-kvm starts up with a miniature resolution by default. 640x480 - on my 1680x1050 laptop screen. It's so small that initially i even overlooked that i started it. It should multiplex pixels up to a reasonable screen size by default.

No virtualization emulator today does this, it is not a reasonable thing to do. Unless you are running compiz and use hardware scaling. We should look into it.

Quoted 9 lines · Avi Kivity

Quoted 7 lines · Ingo Molnar

Nobody is really 'in charge' of how KVM gets delivered to the user. You isolated the fun kernel part for you and pushed out the boring bits to user-space. So if mundane things like mouse integration sucks 'hey that's a user-space tooling problem', if file integration sucks then 'hey, that's an admin problem', if it cannot be used over the network 'hey, that's an Xorg problem', etc. etc.

Quoted 5 lines

btw, mouse integration works with -usbdevice tablet and recent Fedoras, 'it was an X.org driver problem'.

Really, I don't understand your problems.

I run bleeding edge rawhide on my main desktop so i just tried latest& greatest KVM and Qemu bits and started up kvm-qemu with some Fedora and XP images i had around:

   2.6.33-0.44.rc8.git0.fc13.x86_64
   qemu-system-x86-0.12.2-6.fc13.x86_64

Here's my experience with it:

- qemu-kvm starts up with a miniature resolution by default. 640x480 - on my

1680x1050 laptop screen. It's so small that initially i even overlooked that i started it. It should multiplex pixels up to a reasonable screen size by default.

- The mouse is trapped straight away by default if you click into it. That's very annoying if you actually try to integrate a guest OS into your desktop
- it's not just 'another, slightly weird app' but a sticky, opinionated GUI component that you have to fight with all the time.

- Once trapped it's not obvious how to untrap the mouse. The qemu window title says:

QEMU: Press Ctrl-ALT to exit grab

Of course once you _know_ what a 'grab' is, you'll know where to look. At minimum it should say:

QEMU: Press Ctrl-ALT to exit mouse grab

But to first-time users it's an annoying trap of the mouse and with no obvious place to look for help. [besides, it doesnt tell which Ctrl and which ALT to use - it's the left side. The right side Ctrl does not work.]

- Graphics performance is awful even with the 640x480 miniature version. During bootup I can see it drawing single characters. This is a Core2 2.8GHz.

- Sound does not work by default. I have to go dig into command-line options to see that i need to add: "-soundhw all". Why isnt sound enabled by default?

- Qemu images are not integrated into the rest of the desktop. If i click on a Qemu image it says:

Could not display "/home/mingo/qemu/hda.img".

The file is of an unknown type.

10 years of Qemu and its base image format is still 'unknown' ?

- Random bugs. I tried to boot some old Fedora image i had around, it says:

spirit:~/qemu> qemu-kvm ./hda-fedora.img
kvm: unhandled exit 80000021
kvm_run returned -22

- When it boots up, the Qemu window flips around its size crazily, as the BIOS, the bootloader and the OS sets different screen resolutions. To the user that technical detail is immaterial, what matters is an amateurish-looking app that flips its window size as if it was an adware popup window trying to avoid being caught.

- There's no obvious way to activate paravirt drivers on the Windows side. There's no friendly "install guest drivers" button to click on with Qemu.

_Of course_ you will end up emulating hardware in KVM (and passing it through to the guest once it's clear that emulation performance sucks) and sooner or later you will end up requesting unreasonable things of the host kernel to achieve that ...

- Another small detail: there is zero on-screen help (beyond the Ctrl-ALT line) for a newbie to quickly find his way around it. No wiki address, no help, no nothing. There's not even any hint about what this window does. Which guest is it? In what state is that guest?

- But i'm a more advanced user so i dont need help screens, i knew that the "go full screen" hotkey is:

LeftCtrl-LeftALT-F

... except that it is a one-way road: pressing it for a second time does not restore the window, trapping me in the guest altogether! Ctrl-ALT does not exit the trap either. I had to shut down the guest to get back my X desktop.

etc., etc.

( I could go on and on about finer details of good integration, like the
difficulty of integrating host/guest files, networking, no way to quickly attach ISOs via that window, no way to activate networking, sound and no way to snapshot, no way to increase memory size except a command line option. )

etc - but that's not the point really: i only spent 10 minutes on this and i didnt try hard at all - _11_ bugs/annoyances from all across the functionality spectrum.

And the thing is _me_ reporting bugs does not matter at all in this picture, so please dont come with "why didnt you report this?".

_Anyone_ with half a brain who takes a critical look at this virtualization solution would notice the same. Still, it's essentially unchanged from 5 years ago.

Why is that so? I have outlined my opinion that this is due to the artificial package separation / over-modularization and no-one really being in charge of "KVM quality as a whole" - and i'm wondering what your theory is how such a state of affairs became possible.

I'm not trolling you at all: is it _really_ not obvious to you that the KVM/qemu usability status quo honestly sucks, to an unbiased observer?

And AFAICS KVM developers keep concentrating on all the wrong things due to that bad split/packaging: writing newer and newer low level kernel patches and optimizations which are nice but in large part irrelevant because the _fundamental basics_ of usability suck so much ... But to you it's probably just another external package so not really something you can do much about, right?

Really, the KVM design is so nice in many regards and Qemu has come forward leaps and bounds in the past few years as well, how can you miss such basic areas of weakness? 'First impression' is the thing that gives you new developers - it's any OSS project's bread and butter.

Thanks,

Ingo

* Zachary Amsden<[email protected]> wrote:

[...]

Quoted 7 lines

Second, it's not over-modularized. The modules are the individual components of the architecture. How would you propose to put it differently. They really can't naturally combine. And with the code quality of qemu in general being problematic by Linux kernel standards, it's not natural to move the device emulation directly into the kernel module. So this is why we are where we are today.

I'm not talking about moving it into a kernel _module_ - albeit that alone is a worthwile thing to do for any performance sensitive hw component.

I was talking about the option of a clean, stripped down Qemu base hosted in the kernel proper, in linux/tools/kvm/ or so. If i were running a virtualization effort it would be the first place i'd consider to put my tooling into.

It would be a no-brainer: most of the devs come from the KVM side, and KVM itself makes little sense without Qemu, and Qemu makes little sense without KVM these days. (and i know about the non-KVM and non-x86 roots of Qemu - still, it's not a significant piece of usage today)

Quoted 11 lines · Zachary Amsden

Quoted 7 lines

Here's my experience with it:

- qemu-kvm starts up with a miniature resolution by default. 640x480 - on my 1680x1050 laptop screen. It's so small that initially i even overlooked that i started it. It should multiplex pixels up to a reasonable screen size by default.

No virtualization emulator today does this, it is not a reasonable thing to do. Unless you are running compiz and use hardware scaling. We should look into it.

I tried VirtualBox and it does something sane here: while it does not give a larger guest screen area, it at least creates a large enough X window, with any border area whited out.

That looks far more pleasing aesthetically than the Qemu method of resizing the full window wildly. The guest still resizes, but it stays within the same X window and the border is white.

Also, with Windows guests, the Windows side resolution resizes dynamically if the window is grown/shrunk by the user. (I suspect that is done via a paravirt driver on the guest side, through 'Guest Additions'.)

Ingo

On 02/27/2010 04:56 AM, Ingo Molnar wrote:

The other type of user we target is power virtualization/emulation users. We certainly could do better for this type of user but it's never going to fit what your expectation of desktop virtualization is. Qemu is never going to be like running VMware Workstation or VirtualBox.

There is very little split between qemu and kvm from a project perspective. If anything, our lack of focus on desktop virtualization comes from the split between qemu/kvm and libvirt. All of the end-user usability is done in the context of libvirt and virt-manager whereas the qemu/kvm developers tend to focus on performance, features, and robustness.

I think we don't spend enough time in qemu/kvm thinking about how we interact with libvirt with respect to end-to-end usability. It's something I do believe we need to address and it's an area I have been focusing on recently.

* Zachary Amsden<[email protected]> wrote:

[...]

Quoted 7 lines

Second, it's not over-modularized. The modules are the individual components of the architecture. How would you propose to put it differently. They really can't naturally combine. And with the code quality of qemu in general being problematic by Linux kernel standards, it's not natural to move the device emulation directly into the kernel module. So this is why we are where we are today.

I'm not talking about moving it into a kernel _module_ - albeit that alone is a worthwile thing to do for any performance sensitive hw component.

I was talking about the option of a clean, stripped down Qemu base hosted in the kernel proper, in linux/tools/kvm/ or so. If i were running a virtualization effort it would be the first place i'd consider to put my tooling into.

Quoted 8 lines

I'm not talking about moving it into a kernel _module_ - albeit that alone is a worthwile thing to do for any performance sensitive hw component.

I was talking about the option of a clean, stripped down Qemu base hosted in the kernel proper, in linux/tools/kvm/ or so. If i were running a virtualization effort it would be the first place i'd consider to put my tooling into.

So ripping out a clean part interface like PCI bus infrastructure and using it in the kernel, for example, does nothing except put that infrastructure in two different places, because everything the kernel does, userspace will have to do again anyway. So now you have twice as much code involving the same idea and you have to keep the pieces in sync and from trampling each other.

The only parts that warrant such complexity and high risk for bugs are performance critical things like the PIT and APIC.

Em Mon, Mar 01, 2010 at 06:48:07AM -1000, Zachary Amsden escreveu:

Quoted 19 lines · Ingo Molnar

Quoted 10 lines

I'm not talking about moving it into a kernel _module_ - albeit that alone is a worthwile thing to do for any performance sensitive hw component.

I was talking about the option of a clean, stripped down Qemu base hosted in the kernel proper, in linux/tools/kvm/ or so. If i were running a virtualization effort it would be the first place i'd consider to put my tooling into.

So ripping out a clean part interface like PCI bus infrastructure and using it in the kernel, for example, does nothing except put that infrastructure in two different places, because everything the kernel does, userspace will have to do again anyway. So now you have twice as much code involving the same idea and you have to keep the pieces in sync and from trampling each other.

Quoted 3 lines

The only parts that warrant such complexity and high risk for bugs are performance critical things like the PIT and APIC.

I guess there is some misunderstanding here, the tools/ directory that lives in the kernel _sources_, has no kernel source, its all userspace stuff.

Quoted 3 lines

I guess there is some misunderstanding here, the tools/ directory that lives in the kernel _sources_, has no kernel source, its all userspace stuff.

Yeah, no morning coffee :=)

So instead we are advocating forking qemu into the kernel. That makes even less sense. It's not sustainable or eco-friendly to either community.

Here's my experience with it:

 - qemu-kvm starts up with a miniature resolution by default. 640x480 - on my 1680x1050 laptop screen. It's so small that initially i even overlooked that i started it. It should multiplex pixels up to a reasonable screen size by default.

 - The mouse is trapped straight away by default if you click into it. That's very annoying if you actually try to integrate a guest OS into your desktop
  - it's not just 'another, slightly weird app' but a sticky, opinionated GUI component that you have to fight with all the time.

 - Once trapped it's not obvious how to untrap the mouse. The qemu window title says:

         QEMU: Press Ctrl-ALT to exit grab

  Of course once you _know_ what a 'grab' is, you'll know where to look. At minimum it should say:

         QEMU: Press Ctrl-ALT to exit mouse grab

  But to first-time users it's an annoying trap of the mouse and with no obvious place to look for help. [besides, it doesnt tell which Ctrl and which ALT to use - it's the left side. The right side Ctrl does not work.]

 - Graphics performance is awful even with the 640x480 miniature version. During bootup I can see it drawing single characters. This is a Core2 2.8GHz.

 - Sound does not work by default. I have to go dig into command-line options to see that i need to add: "-soundhw all". Why isnt sound enabled by default?

( I could go on and on about finer details of good integration, like the difficulty of integrating host/guest files, networking, no way to quickly attach ISOs via that window, no way to activate networking, sound and no way to snapshot, no way to increase memory size except a command line option. )

I'm not trolling you at all: is it _really_ not obvious to you that the KVM/qemu usability status quo honestly sucks, to an unbiased observer?

Here's our experience with tools/perf/. Hosting the project in the kernel proper helped its quality immensely:

- It's much easier to synchronize new features on the kernel side and on the user-space side. The two go hand in hand - they are often implemented in the same patch.

- It's released together with the kernel, which gives a periodic 3 months release frequency. Not too slow, not too fast.

- Lots of eyeballs and interest. In its mere 8 months of existence tools/perf/ has attracted more than 60 contributors already, and 35 KLOC of new code has been written.

- Code quality requirements are that of the kernel's. No muck allowed and it's not hard to explain what kind of code is preferred.

- Tool breakage bisection is a breeze: there's never any mismatch between tools/perf and the kernel counterpart. With a separate package we'd have more complex test and bisection scenarios.

- Code distribution is easy: it comes with the kernel. This spreads the code far and wide. It's easy for kernel developers to jump in and help out, the latest devel code is always there, a single 'cd tools/perf/; make -j install' command away.
- Code reuse: we started sharing/librarizing code with the kernel: bitmap.h, hash.h, list.h, rbtree.h, bitops.h, prefetch.h.

etc.

In the KVM context this was obviously only a suggestion though. If i were hacking on kvm-qemu i wouldnt hesitate for a moment to do it: the project has very close ties to kernel-KVM and repo level unification would create various synergies - but you are hacking on it, not me ;-)

If i were doing it i'd probably start with a cleaned up and stripped down version of Qemu, to make it eligible for mainline kernel inclusion.

Ingo

kvm -m 512 -cdrom
/home/kirkland/.cache/testdrive/iso/lucid-desktop-amd64.iso -drive file=/home/kirkland/.cache/testdrive/img/testdrive-disk-0086OD.img,if=virtio,index=0,boot=on -usb -usbdevice tablet -net nic,model=virtio -net user -soundhw es1370

Among these:
* 512MB is a nice step up from the 128MB by default (this one scales
based on your hosts memory)

* virtio is used for disk and network for nice performance gains

* usb device tablet greatly improves the mouse experience

* sound card added

* qcow2 sparse disk image

* desktop is 1024 x 720

Quoted 10 lines · Ingo Molnar

- Code distribution is easy: it comes with the kernel. This spreads
the code
far and wide. It's easy for kernel developers to jump in and help
out, the
latest devel code is always there, a single 'cd tools/perf/; make
-j install'
command away.
- Code reuse: we started sharing/librarizing code with the kernel:
bitmap.h,
hash.h, list.h, rbtree.h, bitops.h, prefetch.h.

You could argue that any project should be in the kernel for these reasons. I see no reason why something as like KVM should be part of the kernel and udev shouldn't be.

Quoted 8 lines

etc.

In the KVM context this was obviously only a suggestion though. If i were hacking on kvm-qemu i wouldnt hesitate for a moment to do it: the project has very close ties to kernel-KVM and repo level unification would create various synergies - but you are hacking on it, not me ;-)

Quoted 2 lines

If i were doing it i'd probably start with a cleaned up and stripped down version of Qemu, to make it eligible for mainline kernel inclusion.

You should try it. I think you'll find that it's not as obvious thing to do as you think it is.

Quoted 6 lines · Ingo Molnar

Here's our experience with tools/perf/. Hosting the project in the kernel proper helped its quality immensely:

- It's much easier to synchronize new features on the kernel side and on the user-space side. The two go hand in hand - they are often implemented in the same patch.

Kernel features and qemu features usually don't have a great amount of intersect. All of the problems you've described are strictly in the qemu space.

Quoted 3 lines

- It's released together with the kernel, which gives a periodic 3 months release frequency. Not too slow, not too fast.

qemu release range in length from 3-6 months depending on distribution schedules. They are very regular.

Quoted 3 lines

- Lots of eyeballs and interest. In its mere 8 months of existence tools/perf/ has attracted more than 60 contributors already, and 35 KLOC of new code has been written.

In our last release, we had around 100 contributors and about 100 KLOC of code written. We've got a lot of eyeballs and a lot of interest.

Quoted 2 lines

- Code quality requirements are that of the kernel's. No muck allowed and it's not hard to explain what kind of code is preferred.

Code quality is subjective. We have a different coding style.

Quoted 3 lines

- Tool breakage bisection is a breeze: there's never any mismatch between tools/perf and the kernel counterpart. With a separate package we'd have more complex test and bisection scenarios.

KVM has a backwards compatible ABI so there's no such thing as mismatch between user and kernel space.

Quoted 6 lines

- Code distribution is easy: it comes with the kernel. This spreads the code far and wide. It's easy for kernel developers to jump in and help out, the latest devel code is always there, a single 'cd tools/perf/; make -j install' command away.
- Code reuse: we started sharing/librarizing code with the kernel: bitmap.h, hash.h, list.h, rbtree.h, bitops.h, prefetch.h.

You could argue that any project should be in the kernel for these reasons. I see no reason why something as like KVM should be part of the kernel and udev shouldn't be.

From a design POV all 'system/kernel utilities', which make little sense

Quoted 7 lines

etc.

In the KVM context this was obviously only a suggestion though. If i were hacking on kvm-qemu i wouldnt hesitate for a moment to do it: the project has very close ties to kernel-KVM and repo level unification would create various synergies - but you are hacking on it, not me ;-)

Quoted 2 lines

If i were doing it i'd probably start with a cleaned up and stripped down version of Qemu, to make it eligible for mainline kernel inclusion.

You should try it. I think you'll find that it's not as obvious thing to do as you think it is.

Quoted 13 lines · Anthony Liguori

Quoted 10 lines · Ingo Molnar

- Code distribution is easy: it comes with the kernel. This
spreads the code
far and wide. It's easy for kernel developers to jump in and
help out, the
latest devel code is always there, a single 'cd tools/perf/;
make -j install'
command away.
- Code reuse: we started sharing/librarizing code with the
kernel: bitmap.h,
hash.h, list.h, rbtree.h, bitops.h, prefetch.h.

Quoted 3 lines

You could argue that any project should be in the kernel for these reasons. I see no reason why something as like KVM should be part of the kernel and udev shouldn't be.

gcc and the kernel are quite closely coupled, btw.

Quoted 4 lines · Ingo Molnar

 - But i'm a more advanced user so i dont need help screens, i knew that the
   "go full screen" hotkey is:

Quoted 9 lines

           LeftCtrl-LeftALT-F

   ... except that it is a one-way road: pressing it for a second time does
   not restore the window, trapping me in the guest altogether! Ctrl-ALT does
   not exit the trap either. I had to shut down the guest to get back my X desktop.

That's a bug.

IMO that's a bug, not a feature. There should be a lot more interaction between kvm-qemu and KVM: for example Qemu should have a feature to install paravirt drivers in the guest, this would be helped by living in the kernel repo.

Quoted 7 lines

Quoted 4 lines

- It's released together with the kernel, which gives a periodic 3 months release frequency. Not too slow, not too fast.

qemu release range in length from 3-6 months depending on distribution schedules. They are very regular.

The Linux kernel is released every 3 months, +- one week. Our experience is that even 6 months would be (way) too painful for distros.

Quoted 5 lines

Quoted 3 lines

- Code quality requirements are that of the kernel's. No muck allowed and it's not hard to explain what kind of code is preferred.

Code quality is subjective. We have a different coding style.

That's somewhat of a problem when for example a KVM kernel-space developer crosses into Qemu code and back. Two separate styles, etc. I certainly remember a 'culture clash' when going from the kernel into Qemu and back. Different principles, different culture. It's better to standardize.

Quoted 4 lines

Quoted 4 lines

- Tool breakage bisection is a breeze: there's never any mismatch between tools/perf and the kernel counterpart. With a separate package we'd have more complex test and bisection scenarios.

Quoted 3 lines

KVM has a backwards compatible ABI so there's no such thing as mismatch between user and kernel space.

perf too is ABI compatible (between releases) - still bisection is a lot easier because the evolution of a particular feature can be bisected back to.

Btw., KVM certainly ha ABI breakages around 2.6.16(?) when it was added, even of released versions.

Also, within a development version you sure sometimes
iterate a new ABI component, right?

With a time-coherent repository both
intentional and unintentional breakages and variations can be bisected back to as they happened.

This is an unconditional advantage and i made use of it numerous times.

Quoted 3 lines

You should try it. I think you'll find that it's not as obvious thing to do as you think it is.

A few years ago I looked into cleaning up Qemu, when i hacked KVM and Qemu. I also wanted to have a 'qemu light', which is both smaller and cleaner, and still fits to KVM. It didnt look particularly hard back then - but it's certainly not zero amount of work.

Cleanups pay - they make a piece of code both more hackable, more debuggable and more appealing to new developers. (i suspect you have no argument with that notion) Also note that it wasnt me who suggested that Qemu wouldnt fit the kernel standards as-is - it was raised by others in this discussion.

Ingo

Quoted 5 lines · Dustin Kirkland

kvm -m 512 -cdrom
/home/kirkland/.cache/testdrive/iso/lucid-desktop-amd64.iso -drive file=/home/kirkland/.cache/testdrive/img/testdrive-disk-0086OD.img,if=virtio,index=0,boot=on -usb -usbdevice tablet -net nic,model=virtio -net user -soundhw es1370

Quoted 4 lines

Among these:
* 512MB is a nice step up from the 128MB by default (this one scales
based on your hosts memory)

I'm not opposed to bumping from 128M to something else. I'd like to see some indication though other than "seems reasonable" as to what the value should be. For instance, if a Fedora 12 install doesn't work without 256M, that's a solid argument to bump the allocation. Short term, I'd like to see this globally configurable.

Quoted 7 lines · Anthony Liguori

Quoted 5 lines · Dustin Kirkland

kvm -m 512 -cdrom
/home/kirkland/.cache/testdrive/iso/lucid-desktop-amd64.iso -drive file=/home/kirkland/.cache/testdrive/img/testdrive-disk-0086OD.img,if=virtio,index=0,boot=on -usb -usbdevice tablet -net nic,model=virtio -net user -soundhw es1370

Quoted 7 lines

Quoted 5 lines

Among these:
* 512MB is a nice step up from the 128MB by default (this one scales
based on your hosts memory)

I'm not opposed to bumping from 128M to something else. I'd like to see some indication though other than "seems reasonable" as to what the value should be. For instance, if a Fedora 12 install doesn't work without 256M, that's a solid argument to bump the allocation. Short term, I'd like to see this globally configurable.

I keep a patch in the SUSE version for quite some time now that bumps the default to 384 for qemu-kvm. That was the first "round" number where an openSUSE installation worked.

Alex

Quoted 9 lines · Dustin Kirkland

kvm -m 512 -cdrom
/home/kirkland/.cache/testdrive/iso/lucid-desktop-amd64.iso -drive file=/home/kirkland/.cache/testdrive/img/testdrive-disk-0086OD.img,if=virtio,index=0,boot=on -usb -usbdevice tablet -net nic,model=virtio -net user -soundhw es1370

Among these:
* 512MB is a nice step up from the 128MB by default (this one scales
based on your hosts memory)

I'm not opposed to bumping from 128M to something else. I'd like to see some indication though other than "seems reasonable" as to what the value should be. For instance, if a Fedora 12 install doesn't work without 256M, that's a solid argument to bump the allocation. Short term, I'd like to see this globally configurable.

Quoted 2 lines

* virtio is used for disk and network for nice performance gains

The problem with virtio is that very few OSes have the drivers right now. For networking, we can do better, and I've just written a spec for a new feature to attempt to address this[1]. For disk, I think we're stuck with ide as the default for a long time.

Quoted 2 lines

* usb device tablet greatly improves the mouse experience

Defaulting usb to on and defaulting to a usb tablet is a reasonable thing to do IMHO.

Quoted 2 lines

* sound card added

This personally would annoy me but if there was wide consensus that this was right, I'd be okay with it.

Quoted 2 lines

* qcow2 sparse disk image

This is independent of qemu.

Quoted 2 lines

* desktop is 1024 x 720

1024x768 and this is what the default is today anyway.

Quoted 3 lines · Anthony Liguori

Defaulting usb to on and defaulting to a usb tablet is a reasonable thing to do IMHO.

\o/ Definitely a better user experience.

Yes, you are quite correct: udev has been argued to be a prime candidate for tools/. (and some other kernel utilities as well)

The real
question to ask is, why are you using qemu directly instead of using virt-manager?

Here's our experience with tools/perf/. Hosting the project in the kernel proper helped its quality immensely:

- It's much easier to synchronize new features on the kernel side and on the user-space side. The two go hand in hand - they are often implemented in the same patch.

The real
question to ask is, why are you using qemu directly instead of using virt-manager?

I don't see where this argument is leading to. So far there are arguments that qemu/kvm sucks as a desktop virtualization, now suddenly the gui tools are shitty and everything should be done cli , because there's no man pages for virt-manager. Explain.

Quoted 4 lines · Nikolai K. Bochev

I don't see where this argument is leading to. So far there are arguments that qemu/kvm sucks as a desktop virtualization, now suddenly the gui tools are shitty and everything should be done cli , because there's no man pages for virt-manager. Explain.

Lets face it: virt-manager has a big bunch of problems. For starters there is no (gui) way to create a virtual machine other than installing one from a iso image or via pxe. So if you downloaded a disk image and want to boot it -> no cookie for you. Likewise if you have a bunch of already installed guests and want to migrate from $othertool to virt-manager you can't do that easily.

virt-manager builds on top of libvirt, so you always have the option to use other tools (virsh command line shell for example) to get something done which virt-manager doesn't provide a gui for. libvirt can handle everything Peter asked for without problems. It still sucks though. The whole point of a GUI is that you do *not* have to go to libvirt.org to figure how to tweak the virtual machine config xml using "virsh edit $vmname".

From the short exposure i had to the VirtualBox GUI, that app certainly looks

Quoted 4 lines · Nikolai K. Bochev

I don't see where this argument is leading to. So far there are arguments that qemu/kvm sucks as a desktop virtualization, now suddenly the gui tools are shitty and everything should be done cli , because there's no man pages for virt-manager. Explain.

Lets face it: virt-manager has a big bunch of problems. For starters there is no (gui) way to create a virtual machine other than installing one from a iso image or via pxe. So if you downloaded a disk image and want to boot it -> no cookie for you. Likewise if you have a bunch of already installed guests and want to migrate from $othertool to virt-manager you can't do that easily.

Quoted 6 lines · Gerd Hoffmann

Quoted 4 lines · Nikolai K. Bochev

I don't see where this argument is leading to. So far there are arguments that qemu/kvm sucks as a desktop virtualization, now suddenly the gui tools are shitty and everything should be done cli , because there's no man pages for virt-manager. Explain.

Quoted 6 lines

Lets face it: virt-manager has a big bunch of problems. For starters there is no (gui) way to create a virtual machine other than installing one from a iso image or via pxe. So if you downloaded a disk image and want to boot it -> no cookie for you. Likewise if you have a bunch of already installed guests and want to migrate from $othertool to virt-manager you can't do that easily.

The ability to import existing KVM disk images is available in the latest release of virt-manager, targetted for Fedora 13.

Migrating
from another tool like VMWare is a much harder than just importing the disk image, since you have to update the drivers inside the guest, likely reconfigure several config files, etc, etc. There is work on a full v2v tool to automate this task, for both Linux& Windows guests.

I remember about a year ago, someone asserting on the list that -usbdevice tablet was very CPU intensive even when not in use, and should be avoided if mouse support wasn't needed, e.g. on non-graphical VMs. Was that actually a significant hit, and is it still true today?

Quoted 14 lines · Daniel P. Berrange

Quoted 12 lines · Gerd Hoffmann

Quoted 4 lines · Nikolai K. Bochev

I don't see where this argument is leading to. So far there are arguments that qemu/kvm sucks as a desktop virtualization, now suddenly the gui tools are shitty and everything should be done cli , because there's no man pages for virt-manager. Explain.

Lets face it: virt-manager has a big bunch of problems. For starters there is no (gui) way to create a virtual machine other than installing one from a iso image or via pxe. So if you downloaded a disk image and want to boot it -> no cookie for you. Likewise if you have a bunch of already installed guests and want to migrate from $othertool to virt-manager you can't do that easily.

Quoted 2 lines

The ability to import existing KVM disk images is available in the latest release of virt-manager, targetted for Fedora 13.

Glad to hear that. Are these bits in F12 virt-preview already?

Quoted 5 lines

Migrating
from another tool like VMWare is a much harder than just importing the disk image, since you have to update the drivers inside the guest, likely reconfigure several config files, etc, etc. There is work on a full v2v tool to automate this task, for both Linux& Windows guests.

I meant migrating from another qemu management tool, be it some other gui tool or self-baked shell scripts or something else.

Quoted 1 line

Glad to hear that. Are these bits in F12 virt-preview already?

I believe so.

Quoted 6 lines

Quoted 5 lines

Migrating
from another tool like VMWare is a much harder than just importing the disk image, since you have to update the drivers inside the guest, likely reconfigure several config files, etc, etc. There is work on
a full v2v tool to automate this task, for both Linux& Windows guests.

Quoted 2 lines

I meant migrating from another qemu management tool, be it some other gui tool or self-baked shell scripts or something else.

That's pretty much what the 'disk import' could deal with, unless we wanted to provide a way to read the config files from those other qemu mgmt tools directly on a case-by-case basis

I do think tools/X and tools/libc would make quite a bit of sense - this is one of the better design aspects of FreeBSD et al. It's a mistake that it's not being done.

Quoted 3 lines

I do think tools/X and tools/libc would make quite a bit of sense - this is one of the better design aspects of FreeBSD et al. It's a mistake that it's not being done.

I don't see what it would buy to have tools/libc. You cannot force users to update kernel-space and user-space in lockstep (Apple forced you to do that sometimes when I used Macs at work, and it was very very inconvenient), so it's not like libc would be able to always assume the latest system calls. There is (relatively) a lot of backwards-compatibility code in libc; it's ugly code, but you have to live with it.

Any case in which you need lockstep upgrades of kernel and libc for bisectability is a bug in libc, and I haven't seen it happen enough to be a problem.

Quoted 3 lines · Ingo Molnar

I do think tools/X and tools/libc would make quite a bit of sense - this is one of the better design aspects of FreeBSD et al. It's a mistake that it's not being done.

I don't see what it would buy to have tools/libc. You cannot force users to update kernel-space and user-space in lockstep (Apple forced you to do that sometimes when I used Macs at work, and it was very very inconvenient), so it's not like libc would be able to always assume the latest system calls. There is (relatively) a lot of backwards-compatibility code in libc; it's ugly code, but you have to live with it.

You imply lockstep updates because both are on the same source tree? I don't see why that would be required, there is an ABI contract to be respected no matter where the sources for the signatories live.

I could've done both the kernel and userspace bits in the same patch.

Quoted 3 lines · Arnaldo Carvalho de Melo

You imply lockstep updates because both are on the same source tree? I don't see why that would be required, there is an ABI contract to be respected no matter where the sources for the signatories live.

It's not about ABIs, it's about not being able to rely exclusively on new syscalls or flags (one for all: SOCK_NONBLOCK) because people can run new glibc on old kernels.

Quoted 3 lines

* desktop is 1024 x 720

1024x768 and this is what the default is today anyway.

Quoted 4 lines · Anthony Liguori

Quoted 3 lines

* desktop is 1024 x 720

Quoted 1 line

1024x768 and this is what the default is today anyway.

That was not my experience, as reported in my post a few days ago ("800x600 max resolution"), nor is it the experience reported in the message that kicked off this thread.

I have been able to get a higher resolution, but it was far from automatic.

Quoted 10 lines · Ross Boylan

Quoted 5 lines · Anthony Liguori

Quoted 3 lines

* desktop is 1024 x 720

1024x768 and this is what the default is today anyway.

That was not my experience, as reported in my post a few days ago ("800x600 max resolution"), nor is it the experience reported in the message that kicked off this thread.

Quoted 2 lines

I have been able to get a higher resolution, but it was far from automatic.

It depends on the guest OS version. QEMU exposes a cirrus logic card by defualt,

and given the lack of vsync/hsync info, the Xorg driver will pick 800x600 as the default resolution in absence of any Xorg.conf About 6 months or so back, we got Xorg guys to add a code to the Xorg cirrus driver that looked for the QEMU PCI subsystem ID and if found, defaults to 1024x768 instead.

Of course this is itself still far from optimal
as a user experiance. We really want it to be fully configured to any resolution as easily as the user would do with a real graphics card & monitor.

Quoted 3 lines · Anthony Liguori

The real
question to ask is, why are you using qemu directly instead of using virt-manager?

Because I suspect Ingo, like me, is a command line user, launching a gui to start kvm when there is a kvm command around just sounds daft.

Also, I just installed and tried it, virt-manager is a total piece of shit,

I wouldn't even know how to begin telling it how to start my freshly baked kernel with serial console on stdio and some block image I just created from the gentoo stage3 tarball.

That is, after 5 minutes clicking I have no idea how to even launch an ISO with the thing, I prefer reading the kvm manpage over using some mouse only gui crap like that.

Quoted 9 lines · Peter Zijlstra

Quoted 4 lines · Anthony Liguori

The real
question to ask is, why are you using qemu directly instead of using virt-manager?

Because I suspect Ingo, like me, is a command line user, launching a gui to start kvm when there is a kvm command around just sounds daft.

Quoted 3 lines

Also, I just installed and tried it, virt-manager is a total piece of shit,

That statement is far from being fair. I use virt-manager quite a lot, since I want to keep track of what's going on on KVM virtualization for end users in Fedora. What's shipped with Fedora 12 is pretty decent in many regards, but as in any other software there's plenty of room for improvements.

The biggest problem with virt-manager isn't virt-manager, it's that it is trying to do a nearly intractable task. Because a qemu virtual machine is not a machine at all, just a disk image without the proper metadata to track the important properties of the machine, like what revision of PCI chipset, how many disk controllers the thing is using, what kind of graphics card, etc.

These are all basic things that are left completely undefined by qemu's lack of a top-level configuration file, and it's an inexcusable disgrace.

So virt-manager or any other management tool has the burden of creating and maintaining a bunch of metadata around this workhorse tool called qemu and invoking libvirt to figure out which set of 100,000 blasted command line options to pass on.

That's why it falls short of expectations at times, not because virt-manager is crap, but because there is no well defined, well designed infrastructure for it to manage and the ad-hoc solution here is total crap.

Zach

These are all basic things that are left completely undefined by qemu's lack of a top-level configuration file, and it's an inexcusable disgrace.

Quoted 4 lines · Zachary Amsden

These are all basic things that are left completely undefined by qemu's lack of a top-level configuration file, and it's an inexcusable disgrace.

There is a top-level configuration file for Qemu, at least in the development tree. It's optional, still, but it's there now.

I guess the first step would be to move away from the 'lets support lots of crappy virtualization solutions at once, poorly' model, and pick one good combo (i'd go for Qemu+KVM) and turn it into a heck of an all-around solution. Then all the other combos will catch up as well. (or will wither away)

( Sidenote: i also looked at the VirtualBox kernel driver. Oh my ... i really
shouldnt have! They should migrate to the KVM kernel-side code ASAP ... )

Quoted 7 lines · Ingo Molnar

Here's our experience with tools/perf/. Hosting the project in the kernel proper helped its quality immensely:

- It's much easier to synchronize new features on the kernel side and on the user-space side. The two go hand in hand - they are often implemented in the same patch.

Just look at an example from today, a perf+KVM feature patch posted by Yanmin Zhang:

   http://www.mail-archive.com/[email protected]/msg29770.html

That single patch implements the following "perf kvm" commands:

perf kvm top
perf kvm record
perf kvm report
perf kvm diff

Both the kernel-space and the user-space changes are in that single patch.

Anyone who'd like to try it out can apply it and get an updated kernel plus updated tooling and can start profiling KVM guests straight away. You just check out the kernel, apply the patch and that's it - you can go. It doesnt get any more convenient than that to do development.

Such kind of a unified repository is a powerful concept, and we make use of those aspects of tools/perf/ every day. You could only pry it out of our cold, dead fingers ;-)

Btw., this is one of the things that FreeBSD does right - and i believe it is one of the technical concepts behind Apple's success as well. Apple, with a tenth's of Linux's effective R&D budget can consistently out-develop Linux. I think that's in part due to there not being a strict chinese wall between the Apple kernel, libraries and applications - it's one coherent project where everyone is well-connected to each piece, with no artificial project-cultural boundaries and barriers. People can and do move between those areas of the larger "Apple" project to achieve their goals - regardless of how many components need touching for a given area of interest.

IMHO we should learn from that - while we are good in many areas there's always aspects of Linux that can be improved. But i digress.

perf really is wonderful, but to be really competitive, and usable to more developers, it needs to be in a graphical environment.  I want 'perf report' output to start out collapsed and drill down by clicking on a tree widget. Clicking on a function name opens its definition.  'perf annotate' should display annotations on my editor window, not in a pager.  I should be able to check events on a list, not using 'perf list'.

Hi Avi,

(slightly off-topic)

Quoted 7 lines · Avi Kivity

perf really is wonderful, but to be really competitive, and usable to more developers, it needs to be in a graphical environment. I want 'perf report' output to start out collapsed and drill down by clicking on a tree widget.
Clicking on a function name opens its definition. 'perf annotate' should
display annotations on my editor window, not in a pager. I should be able to check events on a list, not using 'perf list'.

People keep bringing this up but I don't quite agree. Mac OS X has "shark" which is pretty much what you describe above. However, having used both, I prefer perf's simple UI for two reasons: it's much easier to automate perf commands and text-based reports are superior for sharing results (and keeping track of results when doing performance optimizations).

That said, AFAICT, it should be pretty simple to implement a shark-like UI with GTK as current perf code is pretty good fit for that. I've pondered about doing that myself but quite frankly, I don't see any big gains in that.

Quoted 5 lines

- Graphics performance is awful even with the 640x480 miniature
version.
During bootup I can see it drawing single characters. This is a
Core2

    2.8GHz.

That's a bug. Please report it. Graphics performance isn't great, but it should not be that bad.

Quoted 5 lines · Chris Webb

I remember about a year ago, someone asserting on the list that -usbdevice tablet was very CPU intensive even when not in use, and should be avoided if mouse support wasn't needed, e.g. on non-graphical VMs. Was that actually a significant hit, and is it still true today?

It would appear that this is still the case, at least on slower hosts
- on Atom Z530 (1,6GHz), the XP VM uses ~30% CPU when idle with -usbdevice tablet, but only ~4% without it. However, on a faster host (Core2 Quad 2,66GHz), there's practically no difference (Vista x64 VM uses ~1% CPU when idle regardless of -usbdevice tablet).

Quoted 6 lines · Paolo Bonzini

Quoted 4 lines · Ingo Molnar

I do think tools/X and tools/libc would make quite a bit of sense - this is one of the better design aspects of FreeBSD et al. It's a mistake that it's not being done.

Quoted 7 lines

I don't see what it would buy to have tools/libc. You cannot force users to update kernel-space and user-space in lockstep (Apple forced you to do that sometimes when I used Macs at work, and it was very very inconvenient), so it's not like libc would be able to always assume the latest system calls. There is (relatively) a lot of backwards-compatibility code in libc; it's ugly code, but you have to live with it.

If glibc was part of the kernel repo with a klibc alike approach we wouldnt have such problems: the kernel would provide the system library and that's it. You'd not want to upgrade them separately, just like you generally wouldnt want to upgrade your memory management code separately from the scheduler either.

The same argument could be made in a reverse fashion with just about any part of the kernel: 'it would be better to upgrade the ext3 driver separately' - etc. It's similar to all the classic microkernel versus monolithic kernel arguments.

There are costs with increasing size and increasing integration - but fact is that we can manage a 13+ MLOC kernel just fine and the benefits of an integrated approach far outweigh the costs.

In my experience it's far better to have one project for 'infrastructure'
bits: developed, tested and offered as one coherent entity in essence. A bit like how Xorg does it.

These many splintered kernel facilities are historical legacies in essence (from the times when there was no single usable free OS available), and over-modularization has many costs and few advantages.

Quoted 5 lines

Of course this is itself still far from optimal
as a user experiance. We really want it to be fully configured to any resolution as easily as the user would do with a real graphics card& monitor.

Is there some obstacle to getting the virtual monitor to provide configuration info when it's queried? That seems like the most direct solution.

perf really is wonderful, but to be really competitive, and usable to more developers, it needs to be in a graphical environment.  I want 'perf report' output to start out collapsed and drill down by clicking on a tree widget. Clicking on a function name opens its definition.  'perf annotate' should display annotations on my editor window, not in a pager.  I should be able to check events on a list, not using 'perf list'.

Is something like that suitable for tools/perf/?  I think you'll find the intersection of kernel developers and GUI developers to be fairly small.

perf really is wonderful, but to be really competitive, and usable to more developers, it needs to be in a graphical environment. I want 'perf report' output to start out collapsed and drill down by clicking on a tree widget. Clicking on a function name opens its definition. 'perf annotate' should display annotations on my editor window, not in a pager. I should be able to check events on a list, not using 'perf list'.

Quoted 10 lines

perf really is wonderful, but to be really competitive, and usable to more developers, it needs to be in a graphical environment. I want 'perf report' output to start out collapsed and drill down by clicking on a tree widget.
Clicking on a function name opens its definition. 'perf annotate' should
display annotations on my editor window, not in a pager. I should be able to check events on a list, not using 'perf list'.

Is something like that suitable for tools/perf/? I think you'll find the intersection of kernel developers and GUI developers to be fairly small.

The latest versions of Gnome Sysprof use perf and provide a GTK+ tree interface for the profiling output.

However, they are not configurable at all and don't support anything but call graph profiling, unless they added more features very recently.
It would be nice to extend sysprof into a more capable tool, and one that can read perf output files and do so when launched from the command line.

Em Sun, Mar 07, 2010 at 11:35:31AM +0200, Avi Kivity escreveu:

Quoted 8 lines

perf really is wonderful, but to be really competitive, and usable to more developers, it needs to be in a graphical environment. I want 'perf report' output to start out collapsed and drill down by clicking on a tree widget. Clicking on a function name opens its definition. 'perf annotate' should display annotations on my editor window, not in a pager. I should be able to check events on a list, not using 'perf list'.

Do you really think that more kernel developers would use perf more frequently if it had some GUI?

I plan to work on a ncurses tool combining aspects of the existing perf tools, integrating them more, like you suggest above, but even having worked on a pygtk tool that is close to the kernel [1], I'm unsure if doing it using gtk or QT would be something that would entice more developers to use it.

* Zachary Amsden<[email protected]> wrote:

[...]

Quoted 7 lines

Second, it's not over-modularized. The modules are the individual components of the architecture. How would you propose to put it differently. They really can't naturally combine. And with the code quality of qemu in general being problematic by Linux kernel standards, it's not natural to move the device emulation directly into the kernel module. So this is why we are where we are today.

I'm not talking about moving it into a kernel _module_ - albeit that alone is a worthwile thing to do for any performance sensitive hw component.

I was talking about the option of a clean, stripped down Qemu base hosted in the kernel proper, in linux/tools/kvm/ or so. If i were running a virtualization effort it would be the first place i'd consider to put my tooling into.

It would be a no-brainer: most of the devs come from the KVM side, and KVM itself makes little sense without Qemu, and Qemu makes little sense without KVM these days. (and i know about the non-KVM and non-x86 roots of Qemu - still, it's not a significant piece of usage today)

Quoted 6 lines

Third, it's the maintainers upstream who are in charge of KVM quality as a whole - when you are talking about upstream code quality, and the package maintainers who are in charge of KVM quality as a whole - when you are talking about a distro. This is nothing new - it's just a statement of decentralization.

It's certainly nothing new. Nor was the suckage of CVS newup until Git came along and changed the game on a fundamental basis.

Suckage is there to be fought all the time.

Quoted 9 lines · Anthony Liguori

Quoted 7 lines · Ingo Molnar

Here's our experience with tools/perf/. Hosting the project in the kernel proper helped its quality immensely:

- It's much easier to synchronize new features on the kernel side and on the user-space side. The two go hand in hand - they are often implemented in the same patch.

Quoted 4 lines

Kernel features and qemu features usually don't have a great amount of intersect. All of the problems you've described are strictly in the qemu space.

IMO that's a bug, not a feature. There should be a lot more interaction between kvm-qemu and KVM: for example Qemu should have a feature to install paravirt drivers in the guest, this would be helped by living in the kernel repo.

Quoted 6 lines

Quoted 3 lines

- It's released together with the kernel, which gives a periodic 3 months release frequency. Not too slow, not too fast.

qemu release range in length from 3-6 months depending on distribution schedules. They are very regular.

The Linux kernel is released every 3 months, +- one week. Our experience is that even 6 months would be (way) too painful for distros.

Quoted 5 lines

Quoted 3 lines

- Code quality requirements are that of the kernel's. No muck allowed and it's not hard to explain what kind of code is preferred.

Code quality is subjective. We have a different coding style.

That's somewhat of a problem when for example a KVM kernel-space developer crosses into Qemu code and back. Two separate styles, etc. I certainly remember a 'culture clash' when going from the kernel into Qemu and back. Different principles, different culture. It's better to standardize.

Quoted 7 lines

Quoted 4 lines

- Tool breakage bisection is a breeze: there's never any mismatch between tools/perf and the kernel counterpart. With a separate package we'd have more complex test and bisection scenarios.

KVM has a backwards compatible ABI so there's no such thing as mismatch between user and kernel space.

perf too is ABI compatible (between releases) - still bisection is a lot easier because the evolution of a particular feature can be bisected back to.

Btw., KVM certainly ha ABI breakages around 2.6.16(?) when it was added, even of released versions. Also, within a development version you sure sometimes iterate a new ABI component, right? With a time-coherent repository both intentional and unintentional breakages and variations can be bisected back to as they happened.

This is an unconditional advantage and i made use of it numerous times.

Quoted 4 lines

You could argue that any project should be in the kernel for these reasons. I see no reason why something as like KVM should be part of the kernel and udev shouldn't be.

Yes, you are quite correct: udev has been argued to be a prime candidate for tools/. (and some other kernel utilities as well)

From a design POV all 'system/kernel utilities', which make little sense
without the kernel and are license compatible can (and arguably should) move there.

Obviously there's no pressure to do so - it's only an opportunity.

Quoted 7 lines

Quoted 7 lines

etc.

In the KVM context this was obviously only a suggestion though. If i were hacking on kvm-qemu i wouldnt hesitate for a moment to do it: the project has very close ties to kernel-KVM and repo level unification would create various synergies - but you are hacking on it, not me ;-)

Quoted 6 lines

Quoted 3 lines

If i were doing it i'd probably start with a cleaned up and stripped down version of Qemu, to make it eligible for mainline kernel inclusion.

You should try it. I think you'll find that it's not as obvious thing to do as you think it is.

A few years ago I looked into cleaning up Qemu, when i hacked KVM and Qemu. I also wanted to have a 'qemu light', which is both smaller and cleaner, and still fits to KVM. It didnt look particularly hard back then - but it's certainly not zero amount of work.

Cleanups pay - they make a piece of code both more hackable, more debuggable and more appealing to new developers. (i suspect you have no argument with that notion) Also note that it wasnt me who suggested that Qemu wouldnt fit the kernel standards as-is - it was raised by others in this discussion.

That said, AFAICT, it should be pretty simple to implement a shark-like UI with GTK as current perf code is pretty good fit for that. I've pondered about doing that myself but quite frankly, I don't see any big gains in that.

Quoted 8 lines

Em Sun, Mar 07, 2010 at 11:35:31AM +0200, Avi Kivity escreveu:

Quoted 7 lines

perf really is wonderful, but to be really competitive, and usable to more developers, it needs to be in a graphical environment. I want 'perf report' output to start out collapsed and drill down by clicking on a tree widget. Clicking on a function name opens its definition. 'perf annotate' should display annotations on my editor window, not in a pager. I should be able to check events on a list, not using 'perf list'.

Quoted 2 lines

Do you really think that more kernel developers would use perf more frequently if it had some GUI?

Not much. Is perf's target kernel developers exclusively? Who are we writing this kernel for?

No wonder everything is benchmarked using kbuild.

Quoted 5 lines

I plan to work on a ncurses tool combining aspects of the existing perf tools, integrating them more, like you suggest above, but even having worked on a pygtk tool that is close to the kernel [1], I'm unsure if doing it using gtk or QT would be something that would entice more developers to use it.

Even for kernel developers there are advantages in a GUI, namely that features are easily discovered, the amount of information is easily controlled, and in that you can interact (not redo everything from scratch every time you want to change something). The difference between a curses based tool and a true GUI are minimal for this audience.

Quoted 6 lines

Quoted 3 lines

Do you really think that more kernel developers would use perf more frequently if it had some GUI?

Not much. Is perf's target kernel developers exclusively? Who are we writing this kernel for?

No, we aren't writing this tool only for kernel developers exclusively, but that wasn't my question, it was badly formulated, sorry, I shouldn't have included "kernel" in it :-\

Quoted 7 lines

Even for kernel developers there are advantages in a GUI, namely that features are easily discovered, the amount of information is easily controlled, and in that you can interact (not redo everything from scratch every time you want to change something). The difference between a curses based tool and a true GUI are minimal for this audience.

Ok, I agree with you about easier discoverability of features, path shortened from report to annotate to starting the editor right at the line where some event of interest happened,

will try to keep the
routines not much coupled with ncurses, but definetely ncurses will be the first step.

Quoted 1 line

I keep a patch in the SUSE version for quite some time now that bumps the default to 384 for qemu-kvm. That was the first "round" number where an openSUSE installation worked.

If someone works up a patch and tests at least a couple types of guests to confirm that they all install with that number, I'd be happy to apply it (although we need some trickery to support older pc versions).

Quoted 8 lines · Anthony Liguori

Quoted 3 lines

I keep a patch in the SUSE version for quite some time now that bumps the default to 384 for qemu-kvm. That was the first "round" number where an openSUSE installation worked.

If someone works up a patch and tests at least a couple types of guests to confirm that they all install with that number, I'd be happy to apply it (although we need some trickery to support older pc versions).

We should avoid changing defaults. I don't think in this case it matters, since everyone specifies -m anyway, but as a general rule changing defaults = breakage for the unwary. At least make the default part of the machine type to preserve compatibility.

Quoted 10 lines · Avi Kivity

Quoted 8 lines · Anthony Liguori

Quoted 3 lines

I keep a patch in the SUSE version for quite some time now that bumps the default to 384 for qemu-kvm. That was the first "round" number where an openSUSE installation worked.

If someone works up a patch and tests at least a couple types of guests to confirm that they all install with that number, I'd be happy to apply it (although we need some trickery to support older pc versions).

Quoted 4 lines

We should avoid changing defaults. I don't think in this case it matters, since everyone specifies -m anyway, but as a general rule changing defaults = breakage for the unwary. At least make the default part of the machine type to preserve compatibility.

In that case, Alex, where can I find your +384M patch, because I'd like to carry the same one in Ubuntu...

Quoted 3 lines · Anthony Liguori

Quoted 1 line

I keep a patch in the SUSE version for quite some time now that bumps the default to 384 for qemu-kvm. That was the first "round" number where an openSUSE installation worked.

If someone works up a patch and tests at least a couple types of guests to confirm that they all install with that number, I'd be happy to apply it (although we need some trickery to support older pc versions).

We should avoid changing defaults.

Quoted 18 lines · Dustin Kirkland

Quoted 16 lines · Avi Kivity

Quoted 9 lines · Anthony Liguori

Quoted 4 lines

I keep a patch in the SUSE version for quite some time now that bumps the default to 384 for qemu-kvm. That was the first "round" number where an openSUSE installation worked.

If someone works up a patch and tests at least a couple types of guests to confirm that they all install with that number, I'd be happy to apply it (although we need some trickery to support older pc versions).

We should avoid changing defaults. I don't think in this case it matters, since everyone specifies -m anyway, but as a general rule changing defaults = breakage for the unwary. At least make the default part of the machine type to preserve compatibility.

Quoted 3 lines

In that case, Alex, where can I find your +384M patch, because I'd like to carry the same one in Ubuntu...

It's all in the openSUSE build service. The direct access URL (login required FWIW) is here:

https://build.opensuse.org/package/view_file?file=kvm-qemu-default-memsize.patch&package=kvm&project=Virtualization

It merely changes DEFAULT_RAM_SIZE in vl.c from 128 to 384 for the "kvm" package.

Alex
--
To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html

Quoted 5 lines

It's all in the openSUSE build service. The direct access URL (login required FWIW) is here:

https://build.opensuse.org/package/view_file?file=kvm-qemu-default-memsize.patch&package=kvm&project=Virtualization

Quoted 1 line

It merely changes DEFAULT_RAM_SIZE in vl.c from 128 to 384 for the "kvm" package.

We should attempt to do three things with default ram size:

1) bump it up to a more reasonable number
2) make it specified in the global default config
3) make sure we can provide compatibility support for older machine types

Quoted 6 lines · Avi Kivity

Quoted 3 lines · Anthony Liguori

Quoted 1 line

I keep a patch in the SUSE version for quite some time now that bumps the default to 384 for qemu-kvm. That was the first "round" number where an openSUSE installation worked.

If someone works up a patch and tests at least a couple types of guests to confirm that they all install with that number, I'd be happy to apply it (although we need some trickery to support older pc versions).

We should avoid changing defaults.

I disagree. IMHO, the defaults should represent our best suggestion for any given release. The compatibility machines make it very easier for a user to fix on a particular version of a machine type.

Quoted 7 lines · Anthony Liguori

Quoted 6 lines

It's all in the openSUSE build service. The direct access URL (login required FWIW) is here:

https://build.opensuse.org/package/view_file?file=kvm-qemu-default-memsize.patch&package=kvm&project=Virtualization

It merely changes DEFAULT_RAM_SIZE in vl.c from 128 to 384 for the "kvm" package.

Quoted 5 lines

We should attempt to do three things with default ram size:

1) bump it up to a more reasonable number
2) make it specified in the global default config
3) make sure we can provide compatibility support for older machine types

It's really sad, the amount of code needed to change a number.

Quoted 14 lines · Avi Kivity

Quoted 12 lines · Anthony Liguori

Quoted 6 lines

It's all in the openSUSE build service. The direct access URL (login required FWIW) is here:

https://build.opensuse.org/package/view_file?file=kvm-qemu-default-memsize.patch&package=kvm&project=Virtualization

It merely changes DEFAULT_RAM_SIZE in vl.c from 128 to 384 for the "kvm" package.

We should attempt to do three things with default ram size:

1) bump it up to a more reasonable number
2) make it specified in the global default config
3) make sure we can provide compatibility support for older machine types

Quoted 1 line

It's really sad, the amount of code needed to change a number.

We don't do enough via a config. If we did, we could just have a 0.12 config version that got frozen over time.

So really, if we can make the mem readable by global config, and we can have machine specific configs, it would simplify the problem in the future so that we just had to bump a number.

Quoted 15 lines · Anthony Liguori

Quoted 14 lines · Avi Kivity

Quoted 12 lines · Anthony Liguori

Quoted 6 lines

It's all in the openSUSE build service. The direct access URL (login required FWIW) is here:

https://build.opensuse.org/package/view_file?file=kvm-qemu-default-memsize.patch&package=kvm&project=Virtualization

It merely changes DEFAULT_RAM_SIZE in vl.c from 128 to 384 for the "kvm" package.

We should attempt to do three things with default ram size:

1) bump it up to a more reasonable number
2) make it specified in the global default config
3) make sure we can provide compatibility support for older machine types

Quoted 5 lines

Quoted 1 line

It's really sad, the amount of code needed to change a number.

We don't do enough via a config. If we did, we could just have a 0.12 config version that got frozen over time.

So really, if we can make the mem readable by global config, and we can have machine specific configs, it would simplify the problem in the future so that we just had to bump a number.

Perhaps a json representation of things. We already have the parser.

Quoted 1 line

Perhaps a json representation of things. We already have the parser.

Please no :-)

We have a config format, QemuOpts ties nicely into it as does qdev. We just need to represent machine information via QemuOpts and tie -m to manipulating the memory assigned to a machine. IOW, instead of:

(machine_init)(ram_addr_t ram_size,
const char *boot_device, const char *kernel_filename, const char *kernel_cmdline, const char *initrd_filename, const char *cpu_model)

It should be:

(machine_init)(QemuOpts *opts);

Then we can have a [machine] section in the config where we describe all of these things.

Quoted 11 lines · Jernej Simončič

Quoted 4 lines · Chris Webb

I remember about a year ago, someone asserting on the list that -usbdevice tablet was very CPU intensive even when not in use, and should be avoided if mouse support wasn't needed, e.g. on non-graphical VMs. Was that actually a significant hit, and is it still true today?

It would appear that this is still the case, at least on slower hosts
- on Atom Z530 (1,6GHz), the XP VM uses ~30% CPU when idle with -usbdevice tablet, but only ~4% without it. However, on a faster host (Core2 Quad 2,66GHz), there's practically no difference (Vista x64 VM uses ~1% CPU when idle regardless of -usbdevice tablet).

Looks like the tablet is set to 100 Hz polling rate. We may be able to get away with 30 Hz or even less (ep_bInterval, in ms, in hw/usb-wacom.c).

Quoted 3 lines

Looks like the tablet is set to 100 Hz polling rate. We may be able to get away with 30 Hz or even less (ep_bInterval, in ms, in hw/usb-wacom.c).

Changing the USB tablet polling interval from 10ms to 100ms in both hw/usb-wacom.c and hw/usb-hid.c made no difference except the an increase
in bInterval shown in lsusb -v in the guest and the hint of jerky mouse movement I expected from setting this value so high. A similar change to the polling interval for the keyboard and mouse also made no difference to their performance impact.

Taking the FRAME_TIMER_FREQ down to 100 in hw/usb-uhci.c does seem to reduce the CPU load quite a bit, but at the expense of making the USB tablet (and presumably all other USB devices) very laggy.

Quoted 11 lines

Quoted 4 lines

Looks like the tablet is set to 100 Hz polling rate. We may be able to get away with 30 Hz or even less (ep_bInterval, in ms, in hw/usb-wacom.c).

Changing the USB tablet polling interval from 10ms to 100ms in both hw/usb-wacom.c and hw/usb-hid.c made no difference except the an increase
in bInterval shown in lsusb -v in the guest and the hint of jerky mouse movement I expected from setting this value so high. A similar change to the polling interval for the keyboard and mouse also made no difference to their performance impact.

The USB HID devices implement the SET_IDLE command, so the polling interval will have no real effect on performance.

My guess is that the overhead you're seeing is entirely from the USB host adapter having to wake up and check the transport descriptor lists. This will only result in the guest being woken if a device actually responds (as mentioned above it should not).

Quoted 2 lines

The USB HID devices implement the SET_IDLE command, so the polling interval will have no real effect on performance.

On a Linux guest (F12), I see 125 USB interrupts per second with no mouse movement, so something is broken (on the guest or host).

Quoted 4 lines

My guess is that the overhead you're seeing is entirely from the USB host adapter having to wake up and check the transport descriptor lists. This will only result in the guest being woken if a device actually responds (as mentioned above it should not).

A quick profile on the host side doesn't show this. Instead, I see a lot of select() overhead.

Surprising as there are ~10 descriptors being
polled, so ~1200 polls per second. Maybe epoll will help here.

Quoted 4 lines

Surprising as there are ~10 descriptors being
polled, so ~1200 polls per second. Maybe epoll will help here.

I'm not sure where you get 1200 from. select will be called once per host wakeup. i.e. if the USB controller is enabled then 1k times per second due to the frame tick.

Are you sure there are actually 10 descriptors being polled? Remember that the nfds argument is the value of the largest fd in the set (+1), not the number of descriptors in the set.