Skip to contentSkip to conversation

Split kvm source tarballs

DISCUSSION Avi Kivity 3 participants 19 messages Mar 22, 2009 – Apr 19, 2009 KVM

lore
Share & tools
lore

Avi Kivity

author
#0 lore raw

As part of the preparations for a formal maintenance branch, we're now providing userspace-only source tarballs and kernel module-only source tarballs. The combined kernel/userspace package is still provided.

Userspace package names will take the form 'qemu-kvm-devel-xx'. Kernel module package names will take the form 'kvm-kmod-devel-xx'. The combined package is named, as usual, kvm-xx'.

When maintenance branch packages become available, -devel will be dropped, of course. Only the split packages will be available for the maintenance branch.

The userspace package can run against modules from the matching kernel module package, or against modules from a released Linux kernel (version 2.6.25 or higher). In almost all cases, you can mix and match userspace and kernel -devel packages.

Please test the release candidates in http://userweb.kernel.org/~avi/kvm-85rc1/ for any packaging errors. Suggestions of any kind are also welcome.

Signature · 1 line
error compiling committee.c: too many arguments to function
#1 lore raw
Quoted 11 lines · Avi Kivity

As part of the preparations for a formal maintenance branch, we're now providing userspace-only source tarballs and kernel module-only source tarballs. The combined kernel/userspace package is still provided.

Userspace package names will take the form 'qemu-kvm-devel-xx'. Kernel module package names will take the form 'kvm-kmod-devel-xx'. The combined package is named, as usual, kvm-xx'.

When maintenance branch packages become available, -devel will be dropped, of course. Only the split packages will be available for the maintenance branch.

The userspace package can run against modules from the matching kernel module package, or against modules from a released Linux kernel (version 2.6.25 or higher). In almost all cases, you can mix and match userspace and kernel -devel packages.

Please test the release candidates in http://userweb.kernel.org/~avi/kvm-85rc1/ for any packaging errors. Suggestions of any kind are also welcome.

Hrm, I notice that for qemu-kvm, you basically just pulled out the module source.

What do you think about going a step further and building a proper qemu repository?

Regards,

Anthony Liguori

author
#2 lore raw
Quoted 3 lines · Anthony Liguori

Hrm, I notice that for qemu-kvm, you basically just pulled out the module source.

That's step 1.

Quoted 2 lines

What do you think about going a step further and building a proper qemu repository?

Step 2. I have scripts that do that and will merge with Eduardo's script that also rewrites the qemu commit hashes to correspond to qemu.git on git.kernel.org.

Signature · 1 line
error compiling committee.c: too many arguments to function
#3 lore raw
Quoted 11 lines · Avi Kivity
Quoted 3 lines · Anthony Liguori

Hrm, I notice that for qemu-kvm, you basically just pulled out the module source.

That's step 1.

Quoted 2 lines

What do you think about going a step further and building a proper qemu repository?

Step 2. I have scripts that do that and will merge with Eduardo's script that also rewrites the qemu commit hashes to correspond to qemu.git on git.kernel.org.

That's what I figured. FWIW, the split tarballs work just fine for me.

It may be worth waiting to do step 2 until the IO thread is merged. I think once that happens, we could probably do a sprint to get rid of libkvm in kvm-userspace. That would certainly simplify things.

Regards,

Anthony Liguori

#4 lore raw
Quoted 5 lines · Anthony Liguori

That's what I figured. FWIW, the split tarballs work just fine for me.

It may be worth waiting to do step 2 until the IO thread is merged. I think once that happens, we could probably do a sprint to get rid of libkvm in kvm-userspace. That would certainly simplify things.

Yeah. And having the both common and split repos just confuses the heck out of any user of the repository. I think the right way to split it to wait for libkvm going away and just have a qemu-kvm repository and an entirely separate kernel module repository. It's not like there is anything common but the few exported ABI headers, and we can either keep them in both (would mean qemu-kvm can always build against a defined set of headers) or make qemu-kvm require a kernel source like the current kvm support in upstream qemu.

author
#5 lore raw
Quoted 7 lines · Christoph Hellwig
Quoted 4 lines · Anthony Liguori

That's what I figured. FWIW, the split tarballs work just fine for me.

It may be worth waiting to do step 2 until the IO thread is merged. I think once that happens, we could probably do a sprint to get rid of libkvm in kvm-userspace. That would certainly simplify things.

Quoted 9 lines

Yeah. And having the both common and split repos just confuses the heck out of any user of the repository. I think the right way to split it to wait for libkvm going away and just have a qemu-kvm repository and an entirely separate kernel module repository. It's not like there is anything common but the few exported ABI headers, and we can either keep them in both (would mean qemu-kvm can always build against a defined set of headers) or make qemu-kvm require a kernel source like the current kvm support in upstream qemu.

While I strongly dislike duplicating code under source control, I'm beginning to lean in this direction, since the situation is beginning to confuse me too.

So how about this:

- keep copies of the headers in the qemu repository. 'make sync' becomes a maintainer tool rather than a developer tool
- move qemu to the root of the repository, and reparent libkvm/ user/ and friends under it. this will give us easier merging.
- move the external module kit into kvm.git

We end up with a standalone kvm-userspace.git which is easily mergable to and from qemu.git, and kvm.git which can build either a Linux kernel (and is easily mergable to and from Linus' tree and others) or an external module.

No git submodules or inter-repository dependencies. What's not to like?

Signature · 2 lines
I have a truly marvellous patch that fixes the bug which this
signature is too narrow to contain.
#6 lore raw
Quoted 4 lines · Avi Kivity

So how about this:

- keep copies of the headers in the qemu repository. 'make sync' becomes a maintainer tool rather than a developer tool

Yeah. That similar how we maintain the headers and some shared source file for XFS and libxfs in xfsprogs.

Quoted 2 lines

- move qemu to the root of the repository, and reparent libkvm/ user/ and friends under it. this will give us easier merging.

Yeah. While you're at it user/ might be renamed to something more descriptive.

Quoted 1 line

- move the external module kit into kvm.git

As in your kvm development kernel tree? Not sure it's a good idea to suck this into a full kernel tree. Probably worth making it a small repository of it's own.

author
#7 lore raw
Quoted 7 lines · Christoph Hellwig
Quoted 3 lines

- move qemu to the root of the repository, and reparent libkvm/ user/ and friends under it. this will give us easier merging.

Yeah. While you're at it user/ might be renamed to something more descriptive.

The subdirectory test/ under it can be ported to run under qemu and contributed into qemu's test suite.

Quoted 7 lines
Quoted 2 lines

- move the external module kit into kvm.git

As in your kvm development kernel tree? Not sure it's a good idea to suck this into a full kernel tree. Probably worth making it a small repository of it's own.

If the repo contains only the kit (external-module.h and the hack scripts) we'll be left with dual repositories with their confusion and unbisectability. If the repo contains both the kit and the code, I'll need to commit every kvm.git change into that repository, which I'm sure to botch now and then.

Do you see any specific problem with dropping kvm-userspace.git/kernel/* under kvm.git/scripts/kvm/?

Signature · 1 line
error compiling committee.c: too many arguments to function
#8 lore raw
Quoted 8 lines · Avi Kivity

If the repo contains only the kit (external-module.h and the hack scripts) we'll be left with dual repositories with their confusion and unbisectability. If the repo contains both the kit and the code, I'll need to commit every kvm.git change into that repository, which I'm sure to botch now and then.

Do you see any specific problem with dropping kvm-userspace.git/kernel/* under kvm.git/scripts/kvm/?

Adding more stuff to a kernel tree that doesn't apply to building that tree seems rather odd. But if it makes your life easier go for it, it's defintively much better than the current setup.

#10 lore raw
Quoted 2 lines · Christoph Hellwig

Any idea when the split will happen?

AFAIK, shortly after kvm-85 is released.

Regards,

Anthony Liguori

author
#11 lore raw
In reply to Christoph Hellwig
Quoted 2 lines · Christoph Hellwig

Any idea when the split will happen?

I'm working on it, but hit a snag when rewriting upstream qemu commits to match the official qemu.git:
- my repo and qemu.git used different -k options ("$keyword $" squashing), so I can't match tree objects
- some non-UTF-8 characters got corrupted, so I can't match log messages (some are dups anyway)
- I can't match the git-svn-id line since I ripped it off in kvm-userspace.git
- I can't match the ordinal position of the commit in the changelog since some bogus commits made their way into kvm-userspace.git

I expect I'll work around these issues, but it isn't like I have my script ready to roll.

Signature · 1 line
Do not meddle in the internals of kernels, for they are subtle and quick to panic.
#12 lore raw
In reply to Avi Kivity

Hi Avi,

I spent some time today putting together an approximation of the KVM stable release based on QEMU 0.10.x. In principle, it's not too bad at all because it's just a matter of creating a branch in kvm-userspace that's based a kvm commit sometime after the QEMU 0.10.x release but before the next qemu-cvs merge you did that happened post QEMU 0.10.x. Basically, it was a merge of 72ee81f and ddecccc.

The problem for me is pulling in the QEMU stable fixes. You don't have a branch that tracks the QEMU stable tree and I can pull in the QEMU stable git tree without rewriting history since the directory layout is different.

But if you created a qemu-svn-stable branch that followed the QEMU stable tree in kvm-userspace, like the qemu-cvs branch follows trunk, then it would be pretty easy to create and maintain a kvm_stable_0_10 branch of whatever you'd like to call it in kvm-userspace.

Any chance you could do this? I suspect it's just a matter of creating the branch based off of the qemu-cvs tree at ddecccc and then doing a git-svn fetch.

Regards,

Anthony Liguori

#13 lore raw
Quoted 8 lines · Anthony Liguori

But if you created a qemu-svn-stable branch that followed the QEMU stable tree in kvm-userspace, like the qemu-cvs branch follows trunk, then it would be pretty easy to create and maintain a kvm_stable_0_10 branch of whatever you'd like to call it in kvm-userspace.

Any chance you could do this? I suspect it's just a matter of creating the branch based off of the qemu-cvs tree at ddecccc and then doing a git-svn fetch.

Slightly offtopic, but I always wondered why qemu is hosted in svn. For all project having semi-forks of qemu that they try to keep in sync or even merge back a distributed scm would work so much better.

#14 lore raw
Quoted 11 lines · Christoph Hellwig
Quoted 4 lines · Anthony Liguori

But if you created a qemu-svn-stable branch that followed the QEMU stable tree in kvm-userspace, like the qemu-cvs branch follows trunk, then it would be pretty easy to create and maintain a kvm_stable_0_10 branch of whatever you'd like to call it in kvm-userspace.

Any chance you could do this? I suspect it's just a matter of creating the branch based off of the qemu-cvs tree at ddecccc and then doing a git-svn fetch.

Slightly offtopic, but I always wondered why qemu is hosted in svn. For all project having semi-forks of qemu that they try to keep in sync or even merge back a distributed scm would work so much better.

I'm going to switch it to git fwiw.

Regards,

Anthony Liguori

#15 lore raw
Quoted 6 lines · Anthony Liguori
Quoted 4 lines

Slightly offtopic, but I always wondered why qemu is hosted in svn. For all project having semi-forks of qemu that they try to keep in sync or even merge back a distributed scm would work so much better.

I'm going to switch it to git fwiw.

If it's soon enough maybe the kvm repository reorganization should wait for it so that the new repository can be cloned from upstream qemu?

#16 lore raw
Quoted 7 lines · Christoph Hellwig
Quoted 5 lines · Anthony Liguori
Quoted 5 lines

Slightly offtopic, but I always wondered why qemu is hosted in svn. For all project having semi-forks of qemu that they try to keep in sync or even merge back a distributed scm would work so much better.

Quoted 6 lines
Quoted 2 lines

I'm going to switch it to git fwiw.

If it's soon enough maybe the kvm repository reorganization should wait for it so that the new repository can be cloned from upstream qemu?

It'll be compatible with the kernel.org git tree so there's no need to wait.

Regards,

Anthony Liguori

author
#17 lore raw
In reply to Anthony Liguori
Quoted 10 lines · Anthony Liguori

Hi Avi,

I spent some time today putting together an approximation of the KVM stable release based on QEMU 0.10.x. In principle, it's not too bad at all because it's just a matter of creating a branch in kvm-userspace that's based a kvm commit sometime after the QEMU 0.10.x release but before the next qemu-cvs merge you did that happened post QEMU 0.10.x. Basically, it was a merge of 72ee81f and ddecccc.

The problem for me is pulling in the QEMU stable fixes. You don't have a branch that tracks the QEMU stable tree and I can pull in the QEMU stable git tree without rewriting history since the directory layout is different.

But if you created a qemu-svn-stable branch that followed the QEMU stable tree in kvm-userspace, like the qemu-cvs branch follows trunk, then it would be pretty easy to create and maintain a kvm_stable_0_10 branch of whatever you'd like to call it in kvm-userspace.

Any chance you could do this? I suspect it's just a matter of creating the branch based off of the qemu-cvs tree at ddecccc and then doing a git-svn fetch.

I pushed this under the old naming convention, as kvm-userspace.git maint/2.6.30. I'll rename it once I convert the repo.

Signature · 1 line
error compiling committee.c: too many arguments to function
#18 lore raw
Quoted 5 lines · Avi Kivity
Quoted 4 lines · Anthony Liguori

Hi Avi,

I spent some time today putting together an approximation of the KVM stable release based on QEMU 0.10.x. In principle, it's not too bad at all because it's just a matter of creating a branch in kvm-userspace that's based a kvm commit sometime after the QEMU 0.10.x release but before the next qemu-cvs merge you did that happened post QEMU 0.10.x. Basically, it was a merge of 72ee81f and ddecccc.

Quoted 10 lines
Quoted 6 lines

The problem for me is pulling in the QEMU stable fixes. You don't have a branch that tracks the QEMU stable tree and I can pull in the QEMU stable git tree without rewriting history since the directory layout is different.

But if you created a qemu-svn-stable branch that followed the QEMU stable tree in kvm-userspace, like the qemu-cvs branch follows trunk, then it would be pretty easy to create and maintain a kvm_stable_0_10 branch of whatever you'd like to call it in kvm-userspace.

Any chance you could do this? I suspect it's just a matter of creating the branch based off of the qemu-cvs tree at ddecccc and then doing a git-svn fetch.

I pushed this under the old naming convention, as kvm-userspace.git maint/2.6.30. I'll rename it once I convert the repo.

Sweet, thanks!

Regards,

Anthony Liguori