Skip to contentSkip to conversation

New kvm-related qemu patch queue

DISCUSSION Avi Kivity 3 participants 8 messages Jan 10, 2010 – Jan 17, 2010 KVM

lore
Share & tools
lore

Avi Kivity

author
#0 lore raw

In order to improve qemu.git kvm integration quality wrt performance, features, and reliability Marcelo and I will begin to maintain a patch queue based on qemu.git containing kvm-related patches. We will review and apply patches to this queue, test them using the same test suite that is used for qemu-kvm.git, and regularly submit them for inclusion in qemu.git, mimicking the relationship between kvm.git and Linus' linux-2.6.git.

One of the problems of qemu.git kvm support is that it is a clean reimplementation, and thus some of the nuances that were carefully ironed out in qemu-kvm.git are lost. To that end, we would like to change the process of adding features as follows:

- first, the feature in qemu-kvm.git master is morphed to a form
suitable for merging into qemu.git
- when that has been accomplished, the feature is broken into patches
and merged into the patch queue

This process ensures that important details are not lost: the morphing step attempts to preserve all the nuances, and it is much easier to spot a behaviour change in a review than to spot a missing behaviour in a new feature. This was successfully used to merge i386 and x86_64 arch support in Linux.

Some qemu-kvm.git features (primarily support for very old kernels) can be dropped, simplifying the convergence process.

Over time, qemu-kvm.git master will converge with qemu.git master and all that will remain is the patch queue.

Features in qemu-kvm.git not present in qemu.git include:
- smp support
- in-kernel irqchip
- in-kernel pit
- tpr patching
- device assignment
- kvm paravirtualization
- boot=on (extboot, or a real option rom)
- test suite
- ia64
- signalfd support

Many others are implemented differently in the two trees.

The patch queue will appear as a branch 'uq/master' (for 'upstream queue); if necessary branches for the stable series will also be created.

In order to get a kvm feature into qemu.git, please observe the following process:

- post a patch series against qemu-kvm.git/master that implements the feature, or changes an existing feature to use qemu.git infrastructure
- once that's merged, post a patch series against qemu-kvm.git/uq/master that brings the same code into qemu.git.

The new branch will be subject to the same automatic testing that qemu-kvm.git is.

Please post patches to both qemu-devel and kvm mailing lists.

This won't work for all features, so we'll have to feel our way and make decisions on a case by case basis.

Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
#1 lore raw
Quoted 13 lines · Avi Kivity

In order to improve qemu.git kvm integration quality wrt performance, features, and reliability Marcelo and I will begin to maintain a patch queue based on qemu.git containing kvm-related patches. We will review and apply patches to this queue, test them using the same test suite that is used for qemu-kvm.git, and regularly submit them for inclusion in qemu.git, mimicking the relationship between kvm.git and Linus' linux-2.6.git.

One of the problems of qemu.git kvm support is that it is a clean reimplementation, and thus some of the nuances that were carefully ironed out in qemu-kvm.git are lost. To that end, we would like to change the process of adding features as follows:

Quoted 5 lines

- first, the feature in qemu-kvm.git master is morphed to a form
suitable for merging into qemu.git
- when that has been accomplished, the feature is broken into
patches and merged into the patch queue

If there is the same feature in qemu-kvm.git and qemu.git whom is morphed to whom and who is merged where? I am confused. We have much of duplicated code between qemu-kvm.git and qemu.git it is often almost, but not exactly the same. Is it really productive to set rules who should be morphed and how at this point?

Quoted 30 lines

This process ensures that important details are not lost: the morphing step attempts to preserve all the nuances, and it is much easier to spot a behaviour change in a review than to spot a missing behaviour in a new feature. This was successfully used to merge i386 and x86_64 arch support in Linux.

Some qemu-kvm.git features (primarily support for very old kernels) can be dropped, simplifying the convergence process.

Over time, qemu-kvm.git master will converge with qemu.git master and all that will remain is the patch queue.

Features in qemu-kvm.git not present in qemu.git include:
- smp support
- in-kernel irqchip
- in-kernel pit
- tpr patching
- device assignment
- kvm paravirtualization
- boot=on (extboot, or a real option rom)
- test suite
- ia64
- signalfd support

Many others are implemented differently in the two trees.

The patch queue will appear as a branch 'uq/master' (for 'upstream queue); if necessary branches for the stable series will also be created.

Quoted 6 lines

In order to get a kvm feature into qemu.git, please observe the following process:

- post a patch series against qemu-kvm.git/master that implements the feature, or changes an existing feature to use qemu.git infrastructure

It was other way around till last week. Why change?

Quoted 12 lines

- once that's merged, post a patch series against qemu-kvm.git/uq/master that brings the same code into qemu.git.

The new branch will be subject to the same automatic testing that qemu-kvm.git is.

Please post patches to both qemu-devel and kvm mailing lists.

This won't work for all features, so we'll have to feel our way and make decisions on a case by case basis.

--
Gleb.

author
#2 lore raw
Quoted 21 lines · Gleb Natapov
Quoted 13 lines · Avi Kivity

In order to improve qemu.git kvm integration quality wrt performance, features, and reliability Marcelo and I will begin to maintain a patch queue based on qemu.git containing kvm-related patches. We will review and apply patches to this queue, test them using the same test suite that is used for qemu-kvm.git, and regularly submit them for inclusion in qemu.git, mimicking the relationship between kvm.git and Linus' linux-2.6.git.

One of the problems of qemu.git kvm support is that it is a clean reimplementation, and thus some of the nuances that were carefully ironed out in qemu-kvm.git are lost. To that end, we would like to change the process of adding features as follows:

Quoted 6 lines

- first, the feature in qemu-kvm.git master is morphed to a form
suitable for merging into qemu.git
- when that has been accomplished, the feature is broken into
patches and merged into the patch queue

Quoted 6 lines

If there is the same feature in qemu-kvm.git and qemu.git whom is morphed to whom and who is merged where? I am confused. We have much of duplicated code between qemu-kvm.git and qemu.git it is often almost, but not exactly the same. Is it really productive to set rules who should be morphed and how at this point?

If the feature is already in both, then morph qemu-kvm.git into what is already in qemu.git. Hopefully anything missing in qemu.git will be discovered while making the changes.

Quoted 9 lines
Quoted 7 lines

In order to get a kvm feature into qemu.git, please observe the following process:

- post a patch series against qemu-kvm.git/master that implements the feature, or changes an existing feature to use qemu.git infrastructure

It was other way around till last week. Why change?

There were a lot of regressions.

Signature · 1 line
error compiling committee.c: too many arguments to function
#3 lore raw
Quoted 26 lines · Avi Kivity
Quoted 19 lines · Gleb Natapov
Quoted 13 lines · Avi Kivity

In order to improve qemu.git kvm integration quality wrt performance, features, and reliability Marcelo and I will begin to maintain a patch queue based on qemu.git containing kvm-related patches. We will review and apply patches to this queue, test them using the same test suite that is used for qemu-kvm.git, and regularly submit them for inclusion in qemu.git, mimicking the relationship between kvm.git and Linus' linux-2.6.git.

One of the problems of qemu.git kvm support is that it is a clean reimplementation, and thus some of the nuances that were carefully ironed out in qemu-kvm.git are lost. To that end, we would like to change the process of adding features as follows:

Quoted 5 lines

- first, the feature in qemu-kvm.git master is morphed to a form
suitable for merging into qemu.git
- when that has been accomplished, the feature is broken into
patches and merged into the patch queue

Quoted 5 lines

If there is the same feature in qemu-kvm.git and qemu.git whom is morphed to whom and who is merged where? I am confused. We have much of duplicated code between qemu-kvm.git and qemu.git it is often almost, but not exactly the same. Is it really productive to set rules who should be morphed and how at this point?

Quoted 4 lines

If the feature is already in both, then morph qemu-kvm.git into what is already in qemu.git. Hopefully anything missing in qemu.git will be discovered while making the changes.

What about bugs that are present only in qemu.git? Like this:
http://patchwork.ozlabs.org/patch/42298/. Should it go through qemu-kvm.git/uq/master?

What about this one: http://patchwork.ozlabs.org/patch/42447/ should it be postponed untill qemu.git and qemu-kvm.git converge on using the same cpuid infrastructure, or add similar functionality to qemu-kvm to, or add new cpu flags to qemu-kvm only and when cpuid code converge qemu.git will have it too?

Quoted 10 lines
Quoted 7 lines
Quoted 6 lines

In order to get a kvm feature into qemu.git, please observe the following process:

- post a patch series against qemu-kvm.git/master that implements the feature, or changes an existing feature to use qemu.git infrastructure

It was other way around till last week. Why change?

There were a lot of regressions.

Yeah :(

--
Gleb.

author
#4 lore raw
Quoted 10 lines · Gleb Natapov
Quoted 6 lines

If the feature is already in both, then morph qemu-kvm.git into what is already in qemu.git. Hopefully anything missing in qemu.git will be discovered while making the changes.

What about bugs that are present only in qemu.git? Like this:
http://patchwork.ozlabs.org/patch/42298/. Should it go through qemu-kvm.git/uq/master?

Yes. So there is a central place for kvm patches, and so they see autotest.

Quoted 6 lines

What about this one: http://patchwork.ozlabs.org/patch/42447/ should it be postponed untill qemu.git and qemu-kvm.git converge on using the same cpuid infrastructure, or add similar functionality to qemu-kvm to, or add new cpu flags to qemu-kvm only and when cpuid code converge qemu.git will have it too?

Best to make qemu-kvm.git and qemu.git converge first. Duplicating the patch extends the problem. Of course if something is urgent we can bypass the process.

Signature · 1 line
error compiling committee.c: too many arguments to function
#5 lore raw
Quoted 6 lines · Avi Kivity
Quoted 5 lines · Gleb Natapov
Quoted 5 lines

If the feature is already in both, then morph qemu-kvm.git into what is already in qemu.git. Hopefully anything missing in qemu.git will be discovered while making the changes.

Quoted 6 lines
Quoted 3 lines

What about bugs that are present only in qemu.git? Like this:
http://patchwork.ozlabs.org/patch/42298/. Should it go through qemu-kvm.git/uq/master?

Yes. So there is a central place for kvm patches, and so they see autotest.

So I assume you'll take it then.

Quoted 11 lines
Quoted 6 lines

What about this one: http://patchwork.ozlabs.org/patch/42447/ should it be postponed untill qemu.git and qemu-kvm.git converge on using the same cpuid infrastructure, or add similar functionality to qemu-kvm to, or add new cpu flags to qemu-kvm only and when cpuid code converge qemu.git will have it too?

Best to make qemu-kvm.git and qemu.git converge first. Duplicating the patch extends the problem. Of course if something is urgent we can bypass the process.

Actually, looking into it, this patch is needed for qemu-kvm.git and qemu.git to converge. For qemu-kvm to use kvm -cpu flags the patch below should be applied on top of the previous patch.

Signed-off-by Gleb Natapov <[email protected]>
qemu-kvm-x86.c +1 -1
index 82e362c..b3c371c 100644
@@ -1308,7 +1308,7 @@ int kvm_arch_init_vcpu(CPUState *cenv)
     pv_ent = &cpuid_ent[cpuid_nent++];
     memset(pv_ent, 0, sizeof(*pv_ent));
     pv_ent->function = KVM_CPUID_FEATURES;
-    pv_ent->eax = get_para_features(kvm_context);
+    pv_ent->eax = cenv->cpuid_kvm_features & get_para_features(kvm_context);
 #endif
 
     kvm_trim_features(&cenv->cpuid_features,

--
Gleb.

#6 lore raw
In reply to Avi Kivity
Quoted 1 line · Avi Kivity

In order to improve qemu.git kvm integration quality wrt performance, features, and reliability Marcelo and I will begin to maintain a patch queue based on qemu.git containing kvm-related patches. We will review and apply patches to this queue, test them using the same test suite that is used for qemu-kvm.git, and regularly submit them for inclusion in qemu.git, mimicking the relationship between kvm.git and Linus' linux-2.6.git.

Thanks for setting this up Avi!

I just want to stress that everyone continue CC'ing qemu-devel on all KVM patches. Even if the patch is qemu-kvm specific for the moment, I think it's important for qemu-devel to be exposed to the refactoring work.

It might be good to prefix qemu-kvm.git patches in some manner to make it clear which repository they belong to.

Regards,

Anthony Liguori

author
#7 lore raw
Quoted 8 lines · Anthony Liguori
Quoted 1 line · Avi Kivity

In order to improve qemu.git kvm integration quality wrt performance, features, and reliability Marcelo and I will begin to maintain a patch queue based on qemu.git containing kvm-related patches. We will review and apply patches to this queue, test them using the same test suite that is used for qemu-kvm.git, and regularly submit them for inclusion in qemu.git, mimicking the relationship between kvm.git and Linus' linux-2.6.git.

Thanks for setting this up Avi!

I just want to stress that everyone continue CC'ing qemu-devel on all KVM patches. Even if the patch is qemu-kvm specific for the moment, I think it's important for qemu-devel to be exposed to the refactoring work.

It might be good to prefix qemu-kvm.git patches in some manner to make it clear which repository they belong to.

[patch kvm] - qemu-kvm.git uq/master
[patch qemu-kvm] - qemu-kvm.git master
[patch kvm stable-0.12] - qemu-kvm uq/stable-0.12

etc.

Signature · 1 line
error compiling committee.c: too many arguments to function