Switching to a single qemu tree each per qemu-xen and qemu-trad
RFC 1/1 Switching to a single qemu tree each per qemu-xen and qemu-trad
(CC-ing 2x QEMU maintainers and stable release manager)
The separate trees are a holdover from mercurial, which didn't (at the time) have a good in-repo branching model.
I propose that Xen 4.6 should be the last release which uses these split trees and that instead we combine them into just qemu-xen.git (upstream) and qemu-xen-traditional.git (our old fork), with branches in those repos to match our releases. I picked those names to match the libxl interface names for these.
This will result in one place to get all the qemu stable branches for a given qemu type, without having to remember the naming scheme (which is counter-intuitive). It will allow easier cherry-picking and produce less places for users to look for our code etc.
We could put all branches for both in the same tree but I think we want to just let qemu-xen-trad sit quietly in its own corner. Keeping it separate reinforces that it is almost completely frozen, also it saves me having to think up a branch naming scheme within a single repo.
A proposed mapping for the two tree scheme which broadly follows the xen.git scheme is (paths relative to xenbits git root):
qemu-xen-traditional:
/staging/qemu-xen-unstable.git#master => /qemu-xen-traditional.git#staging /staging/qemu-xen-X.Y-testing.git#master => /qemu-xen-traditional.git#staging-X.Y /qemu-xen-unstable.git#master => /qemu-xen-traditional.git#master /qemu-xen-X.Y-testing.git#master => /qemu-xen-traditional.git#stable-X.Y
NB the revision to use is still referenced in xen.git/Config.mk for each branch, no change here.
XXX why do staging/* exist, and what pushes from staging to the other? Should we ditch one or the other?
qemu-xen:
/staging/qemu-upstream-unstable.git#master => /qemu-xen.git#staging /staging/qemu-upstream-X.Y-testing.git#master => /qemu-xen.git#staging-X.Y /qemu-upstream-unstable.git#master => /qemu-xen.git#master /qemu-upstream-X.Y-testing.git#master => /qemu-xen.git#stable-X.Y
These are push gated by their respective qemu-upstream-* osstest flights.
qemu-mainline:
Upstream remains git://git.qemu.org/qemu.git.
/osstest/qemu.git#mainline/xen-tested-master => /qemu-xen.git#upstream-tested
/osstest/qemu.git should be removed.
NB this is the same qemu-xen.git as above.
Below is an osstest patch which implements this proposed scheme (I've not actually created any trees).
The main open question is what to do about the existing split repos for existing stable branches. We could:
* Teach osstest (ap-push) to push to the old tree as well as the new for existing (<= 4.6) branches only.
* Push a Config.mk update to every stable branch and retire the existing trees on the next relevant point release, if there is one.
In both cases perhaps combined with tweaking gitweb to omit the old names from the project list (http://xenbits.xen.org/gitweb/).
Ian.
Quoted 1 line
From
c87f628a7b505d46f2823252c0c3e3ae0fcb736bMon Sep 17 00:00:00 2001
From: Ian Campbell <[email protected]>
Date: Thu, 30 Jul 2015 13:42:03 +0100
Subject: [PATCH] Switch to merged qemu-xen{,-traditional}.git trees
--- ap-common | 7 +++---- ap-fetch-version | 10 ++++++++-- ap-fetch-version-old | 12 ++++++++---- ap-push | 18 ++++++++++++++---- 4 files changed, 33 insertions(+), 14 deletions(-)
ap-common +3 -4
@@ -26,9 +26,7 @@ : ${TREE_XEN:=git://xenbits.xen.org/xen.git} : ${PUSH_TREE_XEN:=$XENBITS:/home/xen/git/xen.git} -#: ${TREE_QEMU:=git://mariner.uk.xensource.com/qemu-$xenbranch.git} -: ${TREE_QEMU:=git://xenbits.xen.org/staging/qemu-$xenbranch.git} - +: ${TREE_QEMU:=git://xenbits.xen.org/qemu-xen-traditional.git} : ${GIT_KERNEL_ORG:=git://git.kernel.org} : ${KERNEL_SCM:=${GIT_KERNEL_ORG}/pub/scm/linux/kernel/git}
@@ -87,7 +85,8 @@ fi : ${TREEBASE_LINUX_XCP:=http://hg.uk.xensource.com/carbon/trunk/linux-2.6.27} -: ${TREE_QEMU_UPSTREAM:=git://xenbits.xen.org/staging/qemu-upstream-${xenbranch#xen-}.git} +: ${TREE_QEMU_UPSTREAM:=git://xenbits.xen.org/qemu-xen.git} +: ${PUSH_TREE_QEMU_UPSTREAM=$XENBITS:/home/xen/git/qemu-xen.git : ${LOCALREV_QEMU_UPSTREAM:=daily-cron.$branch} : ${TREE_QEMU_MAINLINE:=git://git.qemu.org/qemu.git}
ap-fetch-version +8 -2
@@ -53,9 +53,15 @@ qemu-mainline) repo_tree_rev_fetch_git $branch \ $TREE_QEMU_MAINLINE master $LOCALREV_QEMU_UPSTREAM ;; -qemu-upstream-*) +qemu-upstream-unstable) repo_tree_rev_fetch_git $branch \ - $TREE_QEMU_UPSTREAM master $LOCALREV_QEMU_UPSTREAM + $TREE_QEMU_UPSTREAM staging $LOCALREV_QEMU_UPSTREAM + ;; +qemu-upstream-*-testing) + branchcore=${branch#qemu-upstream-} + branchcore=${branchcore%-testing} + repo_tree_rev_fetch_git $branch \ + $TREE_QEMU_UPSTREAM staging-$branchcore $LOCALREV_QEMU_UPSTREAM ;; linux) repo_tree_rev_fetch_git linux \
ap-fetch-version-old +8 -4
@@ -32,8 +32,6 @@ select_xenbranch : ${BASE_LOCALREV_SEABIOS:=daily-cron.$branch.old} : ${BASE_LOCALREV_OVMF:=daily-cron.$branch.old} -: ${BASE_TREE_QEMU_UPSTREAM:=${TREE_QEMU_UPSTREAM/\/staging\//\/}} - if info_linux_tree "$branch"; then repo_tree_rev_fetch_git linux \ $BASE_TREE_LINUX_THIS $BASE_TAG_LINUX_THIS $BASE_LOCALREV_LINUX
@@ -61,9 +59,15 @@ qemu-mainline) repo_tree_rev_fetch_git $branch \ $BASE_TREE_QEMU_MAINLINE mainline/xen-tested-master $LOCALREV_QEMU_MAINLINE ;; -qemu-upstream-*) +qemu-upstream-unstable) + repo_tree_rev_fetch_git $branch \ + $TREE_QEMU_UPSTREAM master $LOCALREV_QEMU_UPSTREAM + ;; +qemu-upstream-*-testing) + branchcore=${branch#qemu-upstream-} + branchcore=${branchcore%-testing} repo_tree_rev_fetch_git $branch \ - $BASE_TREE_QEMU_UPSTREAM master $LOCALREV_QEMU_UPSTREAM + $TREE_QEMU_UPSTREAM stable-$branchcore $LOCALREV_QEMU_UPSTREAM ;; linux) repo_tree_rev_fetch_git linux \
ap-push +14 -4
@@ -30,8 +30,7 @@ select_xenbranch . ap-common TREE_LINUX=$PUSH_TREE_LINUX -TREE_QEMU_MAINLINE=$PUSH_TREE_QEMU_MAINLINE -TREE_QEMU_UPSTREAM=$XENBITS:/home/xen/git/qemu-upstream-${xenbranch#xen-}.git +TREE_QEMU_UPSTREAM=$PUSH_TREE_QEMU_UPSTREAM TREE_XEN=$PUSH_TREE_XEN TREE_LIBVIRT=$PUSH_TREE_LIBVIRT TREE_RUMPUSERXEN=$PUSH_TREE_RUMPUSERXEN
@@ -63,12 +62,23 @@ xen-*-testing) ;; qemu-mainline) cd $repos/qemu-mainline - git push $TREE_QEMU_MAINLINE $revision:refs/heads/mainline/xen-tested-master + git push $TREE_QEMU_UPSTREAM $revision:refs/heads/upstream-tested ;; -qemu-upstream-*) +qemu-upstream-unstable) cd $repos/$branch git push $TREE_QEMU_UPSTREAM $revision:master ;; +qemu-upstream-*-testing) + branchcore=${branch#qemu-upstream-} + branchcore=${branchcore%-testing} + cd $repos/$branch + git push $TREE_QEMU_UPSTREAM $revision:stable-$branchcore + # Maybe push to historical stable branches too? + #case "$branch" in + #qemu-upstream-4.[23456]-testing) + #...XXX... + #esac + ;; linux-3.*) cd $repos/linux git push $TREE_LINUX $revision:tested/$branch
Signature · 1 line
2.1.4
Ian Campbell writes ("[RFC for-4.7] Switching to a single qemu tree each per qemu-xen and qemu-trad"):
...
Quoted 1 line
qemu-xen-traditional:
...
Quoted 2 lines
XXX why do staging/* exist, and what pushes from staging to the other? Should we ditch one or the other?
Once upon a time, some of these had their own push gate. They are nowadays always pushed together. The staging branches should be abolished.
Quoted 7 lines
The main open question is what to do about the existing split repos for existing stable branches. We could:
* Teach osstest (ap-push) to push to the old tree as well as the new for existing (<= 4.6) branches only.
* Push a Config.mk update to every stable branch and retire the existing trees on the next relevant point release, if there is one.
We could do both of these, so we have a transitional period.
Ian.
Quoted 4 lines · Ian Jackson
Ian Campbell writes ("[RFC for-4.7] Switching to a single qemu tree each per qemu-xen and qemu-trad"):
...Quoted 1 line
qemu-xen-traditional:
Quoted 8 lines
...
Quoted 3 lines
XXX why do staging/* exist, and what pushes from staging to the
other?
Should we ditch one or the other?Once upon a time, some of these had their own push gate. They are nowadays always pushed together. The staging branches should be abolished.
Thanks, I suspect as much.
Quoted 11 lines
Quoted 9 lines
The main open question is what to do about the existing split repos for existing stable branches. We could:
* Teach osstest (ap-push) to push to the old tree as well as the
new
for existing (<= 4.6) branches only.
* Push a Config.mk update to every stable branch and retire the existing trees on the next relevant point release, if there is
one.We could do both of these, so we have a transitional period.
That could work, yes.
e.g. perhaps do the dual push until the corresponding point release happens?
Quoted 10 lines · Ian Campbell
(CC-ing 2x QEMU maintainers and stable release manager)
The separate trees are a holdover from mercurial, which didn't (at the time) have a good in-repo branching model.
I propose that Xen 4.6 should be the last release which uses these split trees and that instead we combine them into just qemu-xen.git (upstream) and qemu-xen-traditional.git (our old fork), with branches in those repos to match our releases. I picked those names to match the libxl interface names for these.
Very much +1 with my XenServer hat on.
Quoted 14 lines
This will result in one place to get all the qemu stable branches for a given qemu type, without having to remember the naming scheme (which is counter-intuitive). It will allow easier cherry-picking and produce less places for users to look for our code etc.
We could put all branches for both in the same tree but I think we want to just let qemu-xen-trad sit quietly in its own corner. Keeping it separate reinforces that it is almost completely frozen, also it saves me having to think up a branch naming scheme within a single repo.
A proposed mapping for the two tree scheme which broadly follows the xen.git scheme is (paths relative to xenbits git root):
Quoted 6 lines
qemu-xen-traditional:
/staging/qemu-xen-unstable.git#master => /qemu-xen-traditional.git#staging /staging/qemu-xen-X.Y-testing.git#master => /qemu-xen-traditional.git#staging-X.Y /qemu-xen-unstable.git#master => /qemu-xen-traditional.git#master /qemu-xen-X.Y-testing.git#master => /qemu-xen-traditional.git#stable-X.Y
And take the change to drop the other random branches in the repository By my observations:
$ git remote show upstream
* remote upstream
...
Remote branches:
ide-write-cache
iwj.block-extendable-flag
origin
qemu
upstream
vga-reverse-merge
xen.netscriptenvQuoted 6 lines
NB the revision to use is still referenced in
xen.git/Config.mkfor each branch, no change here.XXX why do staging/* exist, and what pushes from staging to the other? Should we ditch one or the other?
This is how staging used to work for the mecurial xen trees. I presume therefore that whatever used to be done for the Xen trees are still being done for the current qemu trees.
Quoted 14 lines
qemu-xen:
/staging/qemu-upstream-unstable.git#master => /qemu-xen.git#staging /staging/qemu-upstream-X.Y-testing.git#master => /qemu-xen.git#staging-X.Y /qemu-upstream-unstable.git#master => /qemu-xen.git#master /qemu-upstream-X.Y-testing.git#master => /qemu-xen.git#stable-X.Y
These are push gated by their respective qemu-upstream-* osstest flights.
qemu-mainline:
Upstream remains
git://git.qemu.org/qemu.git.
Quoted 8 lines
/osstest/qemu.git#mainline/xen-tested-master => /qemu-xen.git#upstream-tested
/osstest/qemu.git should be removed.
NB this is the same qemu-xen.git as above.
Below is an osstest patch which implements this proposed scheme (I've not actually created any trees).
LGTM.
Also suitable adjustments to the landing page at http://xenbits.xen.org/
~Andrew
Andrew Cooper writes ("Re: [Xen-devel] [RFC for-4.7] Switching to a single qemu tree each per qemu-xen and qemu-trad"):
Quoted 14 lines
And take the change to drop the other random branches in the repository By my observations:
$ git remote show upstream * remote upstream ... Remote branches: ide-write-cache iwj.block-extendable-flag origin qemu upstream vga-reverse-merge xen.netscriptenv
Yes, all of that can be got rid of. I'm not even sure how much of that got there.
Ian.
Quoted 8 lines · Ian Campbell
(CC-ing 2x QEMU maintainers and stable release manager)
The separate trees are a holdover from mercurial, which didn't (at the time) have a good in-repo branching model.
I propose that Xen 4.6 should be the last release which uses these split trees and that instead we combine them into just qemu-xen.git (upstream) and qemu-xen-traditional.git (our old fork)
(sorry my question is a bit unrelated)
is there a proposed time frame for the retirement of qemu-xen-traditional? http://wiki.xenproject.org/wiki/QEMU_Upstream only mentions VGA passthrough as a missing feature but maybe it can be fixed for 4.7?
Signature · 1 line
Vitaly
Quoted 13 lines · Vitaly Kuznetsov
Quoted 10 lines · Ian Campbell
(CC-ing 2x QEMU maintainers and stable release manager)
The separate trees are a holdover from mercurial, which didn't (at the time) have a good in-repo branching model.
I propose that Xen 4.6 should be the last release which uses these split
trees and that instead we combine them into just qemu-xen.git (upstream)
and qemu-xen-traditional.git (our old fork)
Quoted 6 lines
(sorry my question is a bit unrelated)
is there a proposed time frame for the retirement of qemu-xen-traditional? http://wiki.xenproject.org/wiki/QEMU_Upstream only mentions VGA passthrough as a missing feature but maybe it can be fixed for 4.7?
AIUI people (from Intel?) are working on that.
Due to Windows activation issues which are triggered by changing the h/w platform (aka QEMU) under the guest we are taking a long tail approach here and not actively looking to remove qemu-xen-traditional so that people with VMs installed under old versions can choose the stick with that h/w platform even on newer Xen.
Eventually, some long time down the line, we may decide that the chances that anyone is still running such a VM is negligible and we can remove it.
In the meantime qemu-xen-traditional is essentially frozen, no new features and only adjustments to e.g. keep up with changes to the control libraries etc are made.
Ian.
Quoted 11 lines · Vitaly Kuznetsov
Quoted 8 lines · Ian Campbell
(CC-ing 2x QEMU maintainers and stable release manager)
The separate trees are a holdover from mercurial, which didn't (at the time) have a good in-repo branching model.
I propose that Xen 4.6 should be the last release which uses these split trees and that instead we combine them into just qemu-xen.git (upstream) and qemu-xen-traditional.git (our old fork)
Quoted 6 lines
(sorry my question is a bit unrelated)
is there a proposed time frame for the retirement of qemu-xen-traditional? http://wiki.xenproject.org/wiki/QEMU_Upstream only mentions VGA passthrough as a missing feature but maybe it can be fixed for 4.7?
We also don't yet have a good story for stub domains; and there are some rather hacky things related to PCI pass-through that still have to be worked around in qemu-upstream. (Namely, hvmloader cannot resize the MMIO hole because it can't tell qemu about the resize; so when passing through a device that doesn't fit in the default hole size, you have to manually specify the size in the config file.)
We're working on stub domains with rump kernels, but we're not there yet.
-George
Quoted 10 lines
Quoted 1 line
Quoted 1 line · On 30.07.15 at 16:22, <[email protected]> wrote:
(CC-ing 2x QEMU maintainers and stable release manager)
The separate trees are a holdover from mercurial, which didn't (at the time) have a good in-repo branching model.
I propose that Xen 4.6 should be the last release which uses these split trees and that instead we combine them into just qemu-xen.git (upstream) and qemu-xen-traditional.git (our old fork), with branches in those repos to match our releases.
Yes please!
Jan