Ideas wiki for GSoC 2010

Quoted 18 lines · Alexander Graf

Quoted 15 lines · Avi Kivity
Quoted 14 lines · Alexander Graf
Quoted 11 lines · Avi Kivity
Quoted 7 lines · Alexander Graf
The list is also still missing a lot of potential mentors for the listed ideas. Let me propose some here :)
== Shared memory transport between guest(s) and host ==
Sounds like Avi would be a good fit. I'm pretty unknowledgeable when it comes to shm.
Not sure what this is.

Quoted 9 lines

Quoted 6 lines
Quoted 2 lines
Cam's shared memory device.
That's plain shared memory among guests (though the host could also participate). "transport" evokes something like virtio rings.
I could mentor it, though I prefer something in kvm, and it looks close to completion.
I agree. Take it off the list then :-).

Fair enough. I'd be willing to take up one of the other suggestions.

Cam

Paolo Bonzini

maintainer

2010-03-11 12:09 UTC #12 lore raw

Quoted 5 lines · Alexander Graf

== Write a C QMP library based on QEMU JSON and QMP code ==
Suggested by Anthony, mentored by Anthony?:) Possible other candidates are Luiz and Kraxel I guess? I haven't really tracked QMP that much.

If you guys are okay with this, I think I could mentor since I followed the design of QMP quite closely (and this is the only one that I think I could do a decent job with).

BTW, it worked out much better for me in the past when the student and mentor were in a similar time zone.

Paolo

Luiz Capitulino

author

2010-03-11 13:20 UTC #13 lore raw

Quoted 10 lines · On Thu, 11 Mar 2010 13:09:37 +0100 Paolo Bonzini <[email protected]> wrote:

Quoted 5 lines · Alexander Graf
== Write a C QMP library based on QEMU JSON and QMP code ==
Suggested by Anthony, mentored by Anthony?:) Possible other candidates are Luiz and Kraxel I guess? I haven't really tracked QMP that much.
If you guys are okay with this, I think I could mentor since I followed the design of QMP quite closely (and this is the only one that I think I could do a decent job with).

Sure.

Quoted 5 lines

BTW, it worked out much better for me in the past when the student and mentor were in a similar time zone.
Paolo

Edgar E. Iglesias

2010-03-13 03:13 UTC #14 lore raw

IMO one of the biggest parts of infrastructure QEMU is missing for embedded stuff is a common (qdev) way of modeling DMA backchannels. Or devide to device channels or whatever you want to call it.

Would be nice if someone could bring this to qemu.

Cheers

Luiz Capitulino

author

2010-03-11 13:16 UTC #15 lore raw

Quoted 3 lines · On Thu, 11 Mar 2010 12:25:24 +0100 Alexander Graf <[email protected]> wrote:

== Write a C QMP library based on QEMU JSON and QMP code ==
Suggested by Anthony, mentored by Anthony? :) Possible other candidates are Luiz and Kraxel I guess? I haven't really tracked QMP that much.

I didn't candidate as a mentor myself because Anthony has a better idea wrt
to the public API.

But I certainly can help with the implementation. I have more two or three
QMP projects to suggest, btw.

Quoted 7 lines

== Add support for guest copy/paste ==
This should probably be folded into the above VNC server improvements. By itself it's just too little of a task.
== Device state visualization ==
Jan, Kraxel? Maybe too small for a task?

I think that whether a task is small or not also depends on the student,
of course that we should not come up with a project that can be easily done in two weeks.

On the other hand, 'not that difficult' tasks can be an excellent project
for those really new to open source and serious development. You know, when you're a starter you spend quite a lot of time reading code and trying things out (and there's nothing wrong with that).

So, for this kind of project the mentor only should take extra care to
choose a student that is really going to learn a lot in the project.

Jes Sorensen

2010-03-12 10:47 UTC #16 lore raw

In reply to Paolo Bonzini

Quoted 5 lines · Alexander Graf

The list is also still missing a lot of potential mentors for the listed ideas. Let me propose some here :)
== Shared memory transport between guest(s) and host ==
Sounds like Avi would be a good fit. I'm pretty unknowledgeable when it comes to shm.

Actually this one sounds a lot like SGI's XPMEM. Not sure if it would make sense to try and use that infrastructure for that, but it might be a good starting point....

Cheers,
Jes

Luiz Capitulino

author

2010-03-11 12:59 UTC #17 lore raw

Quoted 9 lines · On Thu, 11 Mar 2010 10:43:09 +0100 Paolo Bonzini <[email protected]> wrote:

Quoted 7 lines · Avi Kivity
Quoted 3 lines · Luiz Capitulino
2. Do we have kvm-specific projects? Can they be part of the QEMU project or do we need a different mentoring organization for it?
Something really interesting is kvm-assisted tcg. I'm afraid it's a bit too complicated to GSoC.

Quoted 6 lines

I suppose the GSoC ideas wiki page will migrate to a QEMU ideas page in some time, so it's good to have ideas written down.
Also, the selection of projects will be done by members of the community, by grading the student's submissions. The bar would be placed higher for someone who picks a complicated project.

Exactly, we also have a 'skill level' tag, setting it to high should
help and note that we can have from grad students to phd ones.

Alexander Graf

2010-03-11 13:00 UTC #18 lore raw

Quoted 15 lines · Luiz Capitulino

Quoted 12 lines · On Thu, 11 Mar 2010 10:43:09 +0100 Paolo Bonzini <[email protected]> wrote:
Quoted 7 lines · Avi Kivity
Quoted 3 lines · Luiz Capitulino
2. Do we have kvm-specific projects? Can they be part of the QEMU project or do we need a different mentoring organization for it?
Something really interesting is kvm-assisted tcg. I'm afraid it's a bit too complicated to GSoC.
I suppose the GSoC ideas wiki page will migrate to a QEMU ideas page in some time, so it's good to have ideas written down.

Quoted 6 lines

Quoted 3 lines
Also, the selection of projects will be done by members of the community, by grading the student's submissions. The bar would be placed higher for someone who picks a complicated project.
Exactly, we also have a 'skill level' tag, setting it to high should help and note that we can have from grad students to phd ones.

I don't think we should put in a correlation between skill level and degree. I myself only have a Bachelor's degree :-).

Alex

Luiz Capitulino

author

2010-03-11 13:22 UTC #19 lore raw

Quoted 18 lines · On Thu, 11 Mar 2010 14:00:46 +0100 Alexander Graf <[email protected]> wrote:

Quoted 15 lines · Luiz Capitulino
Quoted 12 lines · On Thu, 11 Mar 2010 10:43:09 +0100 Paolo Bonzini <[email protected]> wrote:
Quoted 7 lines · Avi Kivity
Quoted 3 lines · Luiz Capitulino
2. Do we have kvm-specific projects? Can they be part of the QEMU project or do we need a different mentoring organization for it?
Something really interesting is kvm-assisted tcg. I'm afraid it's a bit too complicated to GSoC.
I suppose the GSoC ideas wiki page will migrate to a QEMU ideas page in some time, so it's good to have ideas written down.

Quoted 8 lines

Quoted 6 lines
Quoted 3 lines
Also, the selection of projects will be done by members of the community, by grading the student's submissions. The bar would be placed higher for someone who picks a complicated project.
Exactly, we also have a 'skill level' tag, setting it to high should help and note that we can have from grad students to phd ones.
I don't think we should put in a correlation between skill level and degree. I myself only have a Bachelor's degree :-).

Absolutely, my bad :)

Jamie Lokier

2010-03-11 22:59 UTC #20 lore raw

Quoted 8 lines · Avi Kivity

Quoted 4 lines · Luiz Capitulino
2. Do we have kvm-specific projects? Can they be part of the QEMU project or do we need a different mentoring organization for it?
Something really interesting is kvm-assisted tcg. I'm afraid it's a bit too complicated to GSoC.

Is this simpler: kvm-assisted user-mode emulation (no TCG involved)?

-- Jamie

Paolo Bonzini

maintainer

2010-03-12 09:48 UTC #21 lore raw

Quoted 11 lines · Jamie Lokier

Quoted 8 lines · Avi Kivity
Quoted 4 lines · Luiz Capitulino
2. Do we have kvm-specific projects? Can they be part of the QEMU project or do we need a different mentoring organization for it?
Something really interesting is kvm-assisted tcg. I'm afraid it's a bit too complicated to GSoC.
Is this simpler: kvm-assisted user-mode emulation (no TCG involved)?

Or running TCG-compiled code as a guest so that for example KVM takes care of the softmmu (with hardware-assisted paging possibly, yoohoo). But I have no idea if that is what Avi meant. :-)

Paolo

Gleb Natapov

2010-03-12 09:59 UTC #22 lore raw

Quoted 8 lines · Paolo Bonzini

Quoted 7 lines · Jamie Lokier
Quoted 6 lines · Avi Kivity
Quoted 4 lines · Luiz Capitulino
2. Do we have kvm-specific projects? Can they be part of the QEMU project or do we need a different mentoring organization for it?

Quoted 9 lines

Quoted 4 lines
Quoted 2 lines
Something really interesting is kvm-assisted tcg. I'm afraid it's a bit too complicated to GSoC.
Is this simpler: kvm-assisted user-mode emulation (no TCG involved)?
Or running TCG-compiled code as a guest so that for example KVM takes care of the softmmu (with hardware-assisted paging possibly, yoohoo). But I have no idea if that is what Avi meant. :-)

I think it is.

--
Gleb.

Avi Kivity

2010-03-11 12:27 UTC #23 lore raw

Quoted 3 lines · Luiz Capitulino

2. Do we have kvm-specific projects? Can they be part of the QEMU project or do we need a different mentoring organization for it?

Complete big real mode emulation. I'll add this.

Signature · 1 line

error compiling committee.c: too many arguments to function

Lucas Meneghel Rodrigues

2010-03-11 13:09 UTC #24 lore raw

Quoted 5 lines · Luiz Capitulino

Hi there,
Our wiki page for the Summer of Code 2010 is doing quite well:
http://wiki.qemu.org/Google_Summer_of_Code_2010

Just to let you guys know that I'm going to give a talk at the local university (Unicamp) about kvm autotest, and will spread the word about the qemu and kvm summer of code applications, will incentivate the students to apply for qemu and kvm.

The university was the 2nd overall place on number of student proposals accepted on gsoc for the last couple of years, with an excellent completion rate, so I believe we could have some good work coming out of it.

Quoted 23 lines

Now the most important is:
1. Get mentors assigned to projects. Just put your name and email in the right field. It's ok and even desirable to have two mentors per project, but please remember that mentoring is serious work, more info here:
http://code.google.com/p/google-summer-of-code/wiki/AdviceforMentors
http://gsoc-wiki.osuosl.org/index.php/Main_Page
2. Do we have kvm-specific projects? Can they be part of the QEMU project or do we need a different mentoring organization for it?
3. Fill in the missing information for the suggested project (description, skill level, languages, etc)
I will complete our application tomorrow or on Friday.
PS: I'm CC'ing everyone who suggested projects there, except one or two I couldn't find the email address.

Adam Litke

2010-03-12 15:11 UTC #25 lore raw

Hi Luiz. Around the time when I introduced the new Asynchronous monitor command API we had talked about converting all commands to use this new API so that we can cut down on duplicate code paths and confusing code. I would like to propose this as a GSoC project idea. Do you think it should stand as its own project or should we merge it into your "Convert Monitor commands to the QObject API" project?

Signature · 2 lines

Thanks,
Adam

Luiz Capitulino

author

2010-03-12 15:22 UTC #26 lore raw

Quoted 6 lines · On Fri, 12 Mar 2010 09:11:31 -0600 Adam Litke <[email protected]> wrote:

Hi Luiz. Around the time when I introduced the new Asynchronous monitor command API we had talked about converting all commands to use this new API so that we can cut down on duplicate code paths and confusing code. I would like to propose this as a GSoC project idea. Do you think it should stand as its own project or should we merge it into your "Convert Monitor commands to the QObject API" project?

I think it's a project by itself, but I wonder if it's too easy/short
for GSoC. An experienced programmer can do the conversion plus testing in a day or two. There are probably a number of cleanups and adaptions that can take more, but still seems too short.

Adam Litke

2010-03-12 15:39 UTC #27 lore raw

Quoted 10 lines · Luiz Capitulino

Quoted 6 lines · On Fri, 12 Mar 2010 09:11:31 -0600 Adam Litke <[email protected]> wrote:
Hi Luiz. Around the time when I introduced the new Asynchronous monitor command API we had talked about converting all commands to use this new API so that we can cut down on duplicate code paths and confusing code. I would like to propose this as a GSoC project idea. Do you think it should stand as its own project or should we merge it into your "Convert Monitor commands to the QObject API" project?

Quoted 4 lines

I think it's a project by itself, but I wonder if it's too easy/short
for GSoC. An experienced programmer can do the conversion plus testing in a day or two. There are probably a number of cleanups and adaptions that can take more, but still seems too short.

So given the relatively small scope of this additional work, maybe it should be an additional "stretch" goal to be added to your project. Once the student(s) have gone through the trouble to familiarize themselves with the monitor code, they would be well-positioned to complete this extra bit. How difficult do you imagine it will be to convert the remaining commands over to QObject?

Signature · 2 lines

Thanks,
Adam

Luiz Capitulino

author

2010-03-12 15:56 UTC #28 lore raw

Quoted 11 lines · On Fri, 12 Mar 2010 09:38:47 -0600 Adam Litke <[email protected]> wrote:

Quoted 10 lines · Luiz Capitulino
Quoted 6 lines · On Fri, 12 Mar 2010 09:11:31 -0600 Adam Litke <[email protected]> wrote:
Hi Luiz. Around the time when I introduced the new Asynchronous monitor command API we had talked about converting all commands to use this new API so that we can cut down on duplicate code paths and confusing code. I would like to propose this as a GSoC project idea. Do you think it should stand as its own project or should we merge it into your "Convert Monitor commands to the QObject API" project?

Quoted 7 lines

Quoted 4 lines
I think it's a project by itself, but I wonder if it's too easy/short
for GSoC. An experienced programmer can do the conversion plus testing in a day or two. There are probably a number of cleanups and adaptions that can take more, but still seems too short.
So given the relatively small scope of this additional work, maybe it should be an additional "stretch" goal to be added to your project.

It could be, our we could try to come up with something additional.

Quoted 4 lines

Once the student(s) have gone through the trouble to familiarize themselves with the monitor code, they would be well-positioned to complete this extra bit. How difficult do you imagine it will be to convert the remaining commands over to QObject?

Well, I won't set the goal to convert all of them, because seems
too much work and Anthony has said that we may not want all the handlers available under QMP. So, this has to be discussed (preferably before GSoC starts for students).

Also, I have two other projects that could related to the async
conversion:

- Simplify/Improve the QObject API
- Improve error handling (QError conversion involved)

Anthony Liguori

2010-03-12 16:37 UTC #29 lore raw

Quoted 10 lines · Luiz Capitulino

Quoted 5 lines
Once the student(s) have gone through the trouble to familiarize themselves with the monitor code, they would be well-positioned to complete this extra bit. How difficult do you imagine it will be to convert the remaining commands over to QObject?
Well, I won't set the goal to convert all of them, because seems
too much work and Anthony has said that we may not want all the handlers available under QMP. So, this has to be discussed (preferably before GSoC starts for students).

What I would like to see is a clean break between the human monitor and QMP whereas the human monitor is implemented in terms of QMP.

For instance, the x and xp commands are not very useful for QMP. However, a generic memory read/write API would be pretty useful. The x/xp commands would be implemented in terms of the memory QMP API.

Likewise, the sum command can be implemented in terms of the above API.

Regards,

Anthony Liguori

Quoted 6 lines

Also, I have two other projects that could related to the async
conversion:
- Simplify/Improve the QObject API
- Improve error handling (QError conversion involved)

Luiz Capitulino

author

2010-03-12 17:36 UTC #30 lore raw

Quoted 12 lines · On Fri, 12 Mar 2010 10:36:53 -0600 Anthony Liguori…

Quoted 10 lines · Luiz Capitulino
Quoted 5 lines
Once the student(s) have gone through the trouble to familiarize themselves with the monitor code, they would be well-positioned to complete this extra bit. How difficult do you imagine it will be to convert the remaining commands over to QObject?
Well, I won't set the goal to convert all of them, because seems
too much work and Anthony has said that we may not want all the handlers available under QMP. So, this has to be discussed (preferably before GSoC starts for students).

Quoted 8 lines

What I would like to see is a clean break between the human monitor and QMP whereas the human monitor is implemented in terms of QMP.
For instance, the x and xp commands are not very useful for QMP. However, a generic memory read/write API would be pretty useful. The x/xp commands would be implemented in terms of the memory QMP API.
Likewise, the sum command can be implemented in terms of the above API.

Right, makes sense.

So, we need to convert them all but maybe not make all of them
available under QMP.

Avi Kivity

2010-03-15 12:25 UTC #31 lore raw

Quoted 6 lines · Luiz Capitulino

Hi there,
Our wiki page for the Summer of Code 2010 is doing quite well:
http://wiki.qemu.org/Google_Summer_of_Code_2010

I will add another project - iommu emulation. Could be very useful for doing device assignment to nested guests, which could make testing a lot easier.

Signature · 1 line

error compiling committee.c: too many arguments to function

Joerg Roedel

2010-03-15 12:38 UTC #32 lore raw

Quoted 11 lines · Avi Kivity

Quoted 6 lines · Luiz Capitulino
Hi there,
Our wiki page for the Summer of Code 2010 is doing quite well:
http://wiki.qemu.org/Google_Summer_of_Code_2010
I will add another project - iommu emulation. Could be very useful for doing device assignment to nested guests, which could make testing a lot easier.

Good idea. If there is interest I could help to mentor this project.

Joerg

Avi Kivity

2010-03-15 12:42 UTC #33 lore raw

Quoted 8 lines · Joerg Roedel

Quoted 6 lines · Avi Kivity
Quoted 4 lines · Luiz Capitulino
Hi there,
Our wiki page for the Summer of Code 2010 is doing quite well:

Quoted 9 lines

Quoted 7 lines
Quoted 3 lines
http://wiki.qemu.org/Google_Summer_of_Code_2010
I will add another project - iommu emulation. Could be very useful for doing device assignment to nested guests, which could make testing a lot easier.
Good idea. If there is interest I could help to mentor this project.

Thanks. I volunteered Anthony, but he may be a little overcommitted.

Signature · 1 line

error compiling committee.c: too many arguments to function

Anthony Liguori

2010-03-15 13:14 UTC #34 lore raw

Quoted 7 lines · Avi Kivity

Quoted 6 lines · Joerg Roedel
Quoted 5 lines · Avi Kivity
Quoted 4 lines · Luiz Capitulino
Hi there,
Our wiki page for the Summer of Code 2010 is doing quite well:

Quoted 8 lines

Quoted 6 lines
Quoted 5 lines
Quoted 2 lines
http://wiki.qemu.org/Google_Summer_of_Code_2010
I will add another project - iommu emulation. Could be very useful for doing device assignment to nested guests, which could make testing a lot easier.
Good idea. If there is interest I could help to mentor this project.
Thanks. I volunteered Anthony, but he may be a little overcommitted.

Joerg, feel free to put your name against too.

Regards,

Anthony Liguori

Joerg Roedel

2010-03-15 15:59 UTC #35 lore raw

Quoted 14 lines · Anthony Liguori

Quoted 13 lines · Avi Kivity
Quoted 12 lines · Joerg Roedel
Quoted 11 lines · Avi Kivity
Quoted 6 lines · Luiz Capitulino
Hi there,
Our wiki page for the Summer of Code 2010 is doing quite well:
http://wiki.qemu.org/Google_Summer_of_Code_2010
I will add another project - iommu emulation. Could be very useful for doing device assignment to nested guests, which could make testing a lot
easier.

Quoted 5 lines

Quoted 3 lines
Quoted 1 line
Good idea. If there is interest I could help to mentor this project.
Thanks. I volunteered Anthony, but he may be a little overcommitted.
Joerg, feel free to put your name against too.

[x] Done.

Thanks,
Joerg

Muli Ben-Yehuda

2010-03-15 12:53 UTC #36 lore raw

Quoted 11 lines · Avi Kivity

Quoted 5 lines · Luiz Capitulino
Hi there,
Our wiki page for the Summer of Code 2010 is doing quite well:
http://wiki.qemu.org/Google_Summer_of_Code_2010
I will add another project - iommu emulation. Could be very useful for doing device assignment to nested guests, which could make testing a lot easier.

Our experiments show that nested device assignment is pretty much required for I/O performance in nested scenarios.

Cheers,
Muli

Joerg Roedel

2010-03-15 13:03 UTC #37 lore raw

Quoted 7 lines · Muli Ben-Yehuda

Quoted 6 lines · Avi Kivity
Quoted 4 lines · Luiz Capitulino
Hi there,
Our wiki page for the Summer of Code 2010 is doing quite well:

Quoted 8 lines

Quoted 5 lines
Quoted 1 line
http://wiki.qemu.org/Google_Summer_of_Code_2010
I will add another project - iommu emulation. Could be very useful for doing device assignment to nested guests, which could make testing a lot easier.
Our experiments show that nested device assignment is pretty much required for I/O performance in nested scenarios.

Really? I did a small test with virtio-blk in a nested guest (disk read with dd, so not a real benchmark) and got a reasonable read-performance of around 25MB/s from the disk in the l2-guest.

Joerg

Avi Kivity

2010-03-15 13:11 UTC #38 lore raw

Quoted 5 lines · Joerg Roedel

Quoted 5 lines
Quoted 5 lines
I will add another project - iommu emulation. Could be very useful for doing device assignment to nested guests, which could make testing a lot easier.

Quoted 8 lines

Quoted 3 lines
Our experiments show that nested device assignment is pretty much required for I/O performance in nested scenarios.
Really? I did a small test with virtio-blk in a nested guest (disk read with dd, so not a real benchmark) and got a reasonable read-performance of around 25MB/s from the disk in the l2-guest.

Your guest wasn't doing a zillion VMREADs and VMWRITEs every exit.

I plan to reduce VMREAD/VMWRITE overhead for kvm, but not much we can do for other guests.

Signature · 1 line

error compiling committee.c: too many arguments to function

Anthony Liguori

2010-03-15 13:23 UTC #39 lore raw

Quoted 7 lines · Avi Kivity

Quoted 6 lines · Joerg Roedel
Quoted 6 lines
Quoted 4 lines
I will add another project - iommu emulation. Could be very useful for doing device assignment to nested guests, which could make testing a lot easier.
Our experiments show that nested device assignment is pretty much required for I/O performance in nested scenarios.

Quoted 8 lines

Quoted 4 lines
Really? I did a small test with virtio-blk in a nested guest (disk read with dd, so not a real benchmark) and got a reasonable read-performance of around 25MB/s from the disk in the l2-guest.
Your guest wasn't doing a zillion VMREADs and VMWRITEs every exit.
I plan to reduce VMREAD/VMWRITE overhead for kvm, but not much we can do for other guests.

VMREAD/VMWRITEs are generally optimized by hypervisors as they tend to be costly. KVM is a bit unusual in terms of how many times the instructions are executed per exit.

Regards,

Anthony Liguori

Avi Kivity

2010-03-15 15:07 UTC #40 lore raw

Quoted 13 lines · Anthony Liguori

Quoted 12 lines · Avi Kivity
Quoted 10 lines · Joerg Roedel
Quoted 6 lines
Quoted 4 lines
I will add another project - iommu emulation. Could be very useful for doing device assignment to nested guests, which could make testing a lot easier.
Our experiments show that nested device assignment is pretty much required for I/O performance in nested scenarios.
Really? I did a small test with virtio-blk in a nested guest (disk read with dd, so not a real benchmark) and got a reasonable read-performance of around 25MB/s from the disk in the l2-guest.

Quoted 5 lines

Quoted 3 lines
Your guest wasn't doing a zillion VMREADs and VMWRITEs every exit.
I plan to reduce VMREAD/VMWRITE overhead for kvm, but not much we can do for other guests.
VMREAD/VMWRITEs are generally optimized by hypervisors as they tend to be costly. KVM is a bit unusual in terms of how many times the instructions are executed per exit.

Do you know offhand of any unnecessary read/writes? There's update_cr8_intercept(), but on normal exits, I don't see what else we can remove.

Signature · 1 line

error compiling committee.c: too many arguments to function

Jason

2010-03-16 00:25 UTC #41 lore raw

Quoted 8 lines · Avi Kivity <avi <at> redhat.com> writes:

Quoted 6 lines · Anthony Liguori
Quoted 4 lines · Avi Kivity
Your guest wasn't doing a zillion VMREADs and VMWRITEs every exit.
I plan to reduce VMREAD/VMWRITE overhead for kvm, but not much we can do for other guests.

Quoted 8 lines

Quoted 3 lines
VMREAD/VMWRITEs are generally optimized by hypervisors as they tend to be costly. KVM is a bit unusual in terms of how many times the instructions are executed per exit.
Do you know offhand of any unnecessary read/writes? There's update_cr8_intercept(), but on normal exits, I don't see what else we can remove.

In comparing KVM 2.6.31.6b to XenServer 5.5.0, it seems KVM has fewer overall VMREADs and VMWRITEs, but there are a lot of VMWRITEs to Host FS_SEL, Host GS_SEL, Host FS_BASE, and Host GS_BASE that don't appear in Xen. Also, KVM has a lot of MSR accesses to 0xc0000081-0xc0000084 that Xen doesn't have.

Avi Kivity

2010-03-16 06:12 UTC #42 lore raw

Quoted 4 lines · Jason

In comparing KVM 2.6.31.6b to XenServer 5.5.0, it seems KVM has fewer overall VMREADs and VMWRITEs, but there are a lot of VMWRITEs to Host FS_SEL, Host GS_SEL, Host FS_BASE, and Host GS_BASE that don't appear in Xen.

Ugh, these should definitely be eliminated, they keep writing the same value most of the time.

Quoted 3 lines

Also, KVM has a
lot of MSR accesses to 0xc0000081-0xc0000084 that Xen doesn't have.

These are unavoidable. Those msrs are used for system calls and we need them to keep ordinary userspace going.

2.6.33 should reduce their frequency though.

Usually it doesn't make sense to pass them through, but if we detect the guest is writing them often, we can do so and eliminate the exits.

Signature · 1 line

Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Jason

2010-03-16 16:34 UTC #43 lore raw

Quoted 10 lines · Avi Kivity <avi <at> redhat.com> writes:

Quoted 4 lines · Jason
In comparing KVM 2.6.31.6b to XenServer 5.5.0, it seems KVM has fewer overall VMREADs and VMWRITEs, but there are a lot of VMWRITEs to Host FS_SEL, Host GS_SEL, Host FS_BASE, and Host GS_BASE that don't appear in Xen.
Ugh, these should definitely be eliminated, they keep writing the same value most of the time.

Sorry, I forgot to mention it would also be nice to reduce the unconditional writes to Host CR0 if the TS issue can still be dealt with.

Avi Kivity

2010-03-16 16:46 UTC #44 lore raw

Quoted 9 lines · Jason

Quoted 6 lines · Avi Kivity<avi<at> redhat.com> writes:
Quoted 4 lines · Jason
In comparing KVM 2.6.31.6b to XenServer 5.5.0, it seems KVM has fewer overall VMREADs and VMWRITEs, but there are a lot of VMWRITEs to Host FS_SEL, Host GS_SEL, Host FS_BASE, and Host GS_BASE that don't appear in Xen.

Quoted 7 lines

Quoted 4 lines
Ugh, these should definitely be eliminated, they keep writing the same value most of the time.
Sorry, I forgot to mention it would also be nice to reduce the unconditional writes to Host CR0 if the TS issue can still be dealt with.

I have some ideas on how to deal with that.

Signature · 1 line

error compiling committee.c: too many arguments to function

Anthony Liguori

2010-03-16 01:21 UTC #45 lore raw

Quoted 14 lines · Avi Kivity

Quoted 13 lines · Anthony Liguori
Quoted 12 lines · Avi Kivity
Quoted 10 lines · Joerg Roedel
Quoted 6 lines
Quoted 4 lines
I will add another project - iommu emulation. Could be very useful for doing device assignment to nested guests, which could make testing a lot easier.
Our experiments show that nested device assignment is pretty much required for I/O performance in nested scenarios.
Really? I did a small test with virtio-blk in a nested guest (disk read with dd, so not a real benchmark) and got a reasonable read-performance of around 25MB/s from the disk in the l2-guest.

Quoted 7 lines

Quoted 5 lines
Quoted 3 lines
Your guest wasn't doing a zillion VMREADs and VMWRITEs every exit.
I plan to reduce VMREAD/VMWRITE overhead for kvm, but not much we can do for other guests.
VMREAD/VMWRITEs are generally optimized by hypervisors as they tend to be costly. KVM is a bit unusual in terms of how many times the instructions are executed per exit.
Do you know offhand of any unnecessary read/writes? There's update_cr8_intercept(), but on normal exits, I don't see what else we can remove.

Yeah, there are a number of examples.

vmcs_clear_bits() and vmcs_set_bits() read a field of the VMCS and then immediately writes it. This is unnecessary as the same information could be kept in a shadow variable. In vmx_fpu_activate, we call vmcs_clear_bits() followed immediately by vmcs_set_bits(). which means we're reading GUEST_CR0 twice and writing it twice.

vmx_get_rflags() reads from the VMCS and we frequently call get_rflags() followed by a set_rflags() to update a bit. We also don't cache the value between calls and there's a few spots in the code that make multiple calls.

Regards,

Anthony Liguori

Avi Kivity

2010-03-16 05:50 UTC #46 lore raw

Quoted 19 lines · Anthony Liguori

Quoted 18 lines · Avi Kivity
Quoted 17 lines · Anthony Liguori
Quoted 15 lines · Avi Kivity
Quoted 10 lines · Joerg Roedel
Quoted 6 lines
Quoted 4 lines
I will add another project - iommu emulation. Could be very useful for doing device assignment to nested guests, which could make testing a lot easier.
Our experiments show that nested device assignment is pretty much required for I/O performance in nested scenarios.
Really? I did a small test with virtio-blk in a nested guest (disk read with dd, so not a real benchmark) and got a reasonable read-performance of around 25MB/s from the disk in the l2-guest.
Your guest wasn't doing a zillion VMREADs and VMWRITEs every exit.
I plan to reduce VMREAD/VMWRITE overhead for kvm, but not much we can do for other guests.

Quoted 7 lines

Quoted 3 lines
Quoted 1 line
VMREAD/VMWRITEs are generally optimized by hypervisors as they tend to be costly. KVM is a bit unusual in terms of how many times the instructions are executed per exit.
Do you know offhand of any unnecessary read/writes? There's update_cr8_intercept(), but on normal exits, I don't see what else we can remove.
Yeah, there are a number of examples.
vmcs_clear_bits() and vmcs_set_bits() read a field of the VMCS and then immediately writes it. This is unnecessary as the same information could be kept in a shadow variable. In vmx_fpu_activate, we call vmcs_clear_bits() followed immediately by vmcs_set_bits(). which means we're reading GUEST_CR0 twice and writing it twice.

This should be much better these days (2.6.34-rc1) as vmx_fpu_activate() is called at most once per heavyweight exit (and I have evil plans to reduce it even further). Still, that code should be optimized.

Quoted 1 line

vmx_get_rflags() reads from the VMCS and we frequently call get_rflags() followed by a set_rflags() to update a bit. We also don't cache the value between calls and there's a few spots in the code that make multiple calls.

We definitely should cache that (and segment access from the emulator as well). But I'd have thought this to be relatively infrequent. At least with Linux, using x2apic and virtio allows you to eliminate most emulator access, if you have npt or ept.

Signature · 1 line

Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Joerg Roedel

2010-03-15 13:24 UTC #47 lore raw