Tweaks to allow running non-master production instances

Now that the Cambridge osstest instance is no longer the XenProject master instance it needs some tweaks to allow us to keep using it for local testing.

Since v2 I have now tested this in as close to how the osstest user in Cambridge would, but in my own $HOME/testing.git and it appears to do the right things. As part of that I made various fixes, in particular to the ap-fetch-version stuff.

A test report email from my adhoc flight is below. (the Cambridge VM is current knackered so the link doesn't work and it is only internally available now).

WRT applying this I think normally we should apply such changes to the upstream production instance and let them filter through, but in this case I think it would be better to push my ad-hoc tested branch (based on upstream "cambridge: Set HostProp_DhcpWatchMethod back to woking", reference by pull req below) to the local Cambridge pretest, allow it to automerge upstream master into that and test the result. Once that passes I will send a pull-request to merge the result back into the mainline.

Ian.

The following changes since commit eff0c7780505a610a39f995fb132318bd3f8647b:

cambridge: Set HostProp_DhcpWatchMethod back to woking (2015-04-02 12:03:58 +0100)

are available in the git repository at:

git://xenbits.xen.org/people/ianc/osstest.git non-prod-instances-v2

for you to fetch changes up to 76c499329a6a47117c2c397433b1f950f35dd741:

cambridge: Stop publishing logs to chiark (2015-04-04 08:53:06 +0100)

----------------------------------------------------------------
Ian Campbell (5):
cri-args-hostslists: allow instance specific settings Handle osstest's own local push gate in non-master production instances
cambridge: Add a crontab
cambridge: Do not try to push harness to XenProject instance output
cambridge: Stop publishing logs to chiark

 Osstest.pm                  |  2 ++
 ap-common                   |  2 ++
 ap-fetch-version            | 25 ++++++++++++++++++++++---
 ap-push                     |  8 +++++++-
 cr-ensure-disk-space        |  2 ++
 cr-publish-flight-logs      |  6 ++++--
 cri-args-hostlists          |  4 +++-
 crontab-cambridge           |  5 +++++
 production-config           |  2 ++
 production-config-cambridge |  9 +--------
 10 files changed, 50 insertions(+), 15 deletions(-)
 create mode 100644 crontab-cambridge

flight 37075 osstest play [play]
http://osstest.cam.xci-test.com/~osstest/testlogs/logs/37075/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
test-amd64-amd64-xl-sedf 9 debian-install fail REGR. vs. 36584
test-armhf-armhf-xl-xsm 6 xen-boot fail like 36584
test-amd64-i386-pair 21 guest-migrate/src_host/dst_host fail like 36584

Tests which did not succeed, but are not blocking:

 test-armhf-armhf-xl-multivcpu 18 capture-logs(18)            broken never pass
 test-armhf-armhf-xl          18 capture-logs(18)             broken never pass
 test-armhf-armhf-xl-sedf     18 capture-logs(18)             broken never pass
 test-armhf-armhf-libvirt-xsm  7 capture-logs(7)              broken never pass
 test-armhf-armhf-xl-sedf-pin 18 capture-logs(18)             broken never pass
 test-armhf-armhf-xl-credit2  18 capture-logs(18)             broken never pass
 test-armhf-armhf-xl-midway   18 capture-logs(18)             broken never pass
 test-amd64-amd64-rumpuserxen-amd64 15 rumpuserxen-demo-xenstorels/xenstorels.repeat fail never pass
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm 9 debian-hvm-install fail never pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm 9 debian-hvm-install fail never pass
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm 9 debian-hvm-install fail never pass
 test-amd64-amd64-libvirt-xsm 11 guest-start                  fail   never pass
 test-amd64-amd64-xl-xsm      11 guest-start                  fail   never pass
 test-amd64-i386-xl-xsm       11 guest-start                  fail   never pass
 test-amd64-i386-libvirt-xsm  11 guest-start                  fail   never pass
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm 9 debian-hvm-install fail never pass
 test-amd64-amd64-xl-pvh-amd  11 guest-start                  fail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start                  fail  never pass
 test-amd64-i386-libvirt      12 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-check        fail  never pass
 test-armhf-armhf-xl          12 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-sedf     12 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt-xsm  6 xen-boot                     fail   never pass
 test-armhf-armhf-xl-sedf-pin 12 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-check        fail   never pass
 test-armhf-armhf-libvirt     15 guest-start/debian.repeat    fail   never pass
 test-armhf-armhf-libvirt     12 migrate-support-check        fail   never pass
 test-armhf-armhf-xl-midway   12 migrate-support-check        fail   never pass
 test-amd64-amd64-libvirt     12 migrate-support-check        fail   never pass
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stop              fail never pass
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1 16 guest-stop         fail never pass
 test-amd64-i386-xl-qemut-winxpsp3 16 guest-stop                fail never pass
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1 16 guest-stop         fail never pass
 test-amd64-amd64-xl-qemuu-winxpsp3 16 guest-stop               fail never pass
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop              fail never pass
 test-amd64-i386-xl-win7-amd64 16 guest-stop                   fail  never pass
 test-amd64-amd64-xl-win7-amd64 16 guest-stop                   fail never pass
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop             fail never pass
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop             fail never pass
 test-amd64-i386-xl-qemuu-winxpsp3 16 guest-stop                fail never pass
 test-amd64-i386-xl-winxpsp3-vcpus1 16 guest-stop               fail never pass
 test-amd64-i386-xl-winxpsp3  16 guest-stop                   fail   never pass
 test-amd64-amd64-xl-winxpsp3 16 guest-stop                   fail   never pass

test-amd64-amd64-xl-qemut-winxpsp3 16 guest-stop fail never pass

version targeted for testing:
osstest 2d675fb0de966bae8046b936fe0afe574c6cd9a1
baseline version:
osstest a623769cbac25ae716ff9d8251116f95e2ee2531

jobs:

 build-amd64-xsm                                              pass    
 build-armhf-xsm                                              pass    
 build-i386-xsm                                               pass    
 build-amd64                                                  pass    
 build-armhf                                                  pass    
 build-i386                                                   pass    
 build-amd64-libvirt                                          pass    
 build-armhf-libvirt                                          pass    
 build-i386-libvirt                                           pass    
 build-amd64-pvops                                            pass    
 build-armhf-pvops                                            pass    
 build-i386-pvops                                             pass    
 build-amd64-rumpuserxen                                      pass    
 build-i386-rumpuserxen                                       pass    
 test-amd64-amd64-xl                                          pass    
 test-armhf-armhf-xl                                          broken  
 test-amd64-i386-xl                                           pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64-xsm                fail    
 test-amd64-i386-xl-qemut-debianhvm-amd64-xsm                 fail    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64-xsm                fail    
 test-amd64-i386-xl-qemuu-debianhvm-amd64-xsm                 fail    
 test-amd64-amd64-libvirt-xsm                                 fail    
 test-armhf-armhf-libvirt-xsm                                 broken  
 test-amd64-i386-libvirt-xsm                                  fail    
 test-amd64-amd64-xl-xsm                                      fail    
 test-armhf-armhf-xl-xsm                                      fail    
 test-amd64-i386-xl-xsm                                       fail    
 test-amd64-amd64-xl-pvh-amd                                  fail    
 test-amd64-i386-rhel6hvm-amd                                 pass    
 test-amd64-i386-qemut-rhel6hvm-amd                           pass    
 test-amd64-i386-qemuu-rhel6hvm-amd                           pass    
 test-amd64-amd64-xl-qemut-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemut-debianhvm-amd64                     pass    
 test-amd64-amd64-xl-qemuu-debianhvm-amd64                    pass    
 test-amd64-i386-xl-qemuu-debianhvm-amd64                     pass    
 test-amd64-i386-freebsd10-amd64                              pass    
 test-amd64-amd64-xl-qemuu-ovmf-amd64                         pass    
 test-amd64-i386-xl-qemuu-ovmf-amd64                          pass    
 test-amd64-amd64-rumpuserxen-amd64                           fail    
 test-amd64-amd64-xl-qemut-win7-amd64                         fail    
 test-amd64-i386-xl-qemut-win7-amd64                          fail    
 test-amd64-amd64-xl-qemuu-win7-amd64                         fail    
 test-amd64-i386-xl-qemuu-win7-amd64                          fail    
 test-amd64-amd64-xl-win7-amd64                               fail    
 test-amd64-i386-xl-win7-amd64                                fail    
 test-amd64-amd64-xl-credit2                                  pass    
 test-armhf-armhf-xl-credit2                                  broken  
 test-amd64-i386-freebsd10-i386                               pass    
 test-amd64-i386-rumpuserxen-i386                             pass    
 test-amd64-amd64-xl-pvh-intel                                fail    
 test-amd64-i386-rhel6hvm-intel                               pass    
 test-amd64-i386-qemut-rhel6hvm-intel                         pass    
 test-amd64-i386-qemuu-rhel6hvm-intel                         pass    
 test-amd64-amd64-libvirt                                     pass    
 test-armhf-armhf-libvirt                                     fail    
 test-amd64-i386-libvirt                                      pass    
 test-armhf-armhf-xl-midway                                   broken  
 test-amd64-amd64-xl-multivcpu                                pass    
 test-armhf-armhf-xl-multivcpu                                broken  
 test-amd64-amd64-pair                                        pass    
 test-amd64-i386-pair                                         fail    
 test-amd64-amd64-xl-sedf-pin                                 pass    
 test-armhf-armhf-xl-sedf-pin                                 broken  
 test-amd64-amd64-xl-sedf                                     fail    
 test-armhf-armhf-xl-sedf                                     broken  
 test-amd64-i386-xl-qemut-winxpsp3-vcpus1                     fail    
 test-amd64-i386-xl-qemuu-winxpsp3-vcpus1                     fail    
 test-amd64-i386-xl-winxpsp3-vcpus1                           fail    
 test-amd64-amd64-xl-qemut-winxpsp3                           fail    
 test-amd64-i386-xl-qemut-winxpsp3                            fail    
 test-amd64-amd64-xl-qemuu-winxpsp3                           fail    
 test-amd64-i386-xl-qemuu-winxpsp3                            fail    
 test-amd64-amd64-xl-winxpsp3                                 fail    
 test-amd64-i386-xl-winxpsp3                                  fail    

------------------------------------------------------------
sg-report-flight on osstest.cam.xci-test.com
logs: /home/xc_osstest/logs
images: /home/xc_osstest/images

Logs, config files, etc. are available at
http://osstest.cam.xci-test.com/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary

Pushing revision :

+ branch=osstest
+ revision=2d675fb0de966bae8046b936fe0afe574c6cd9a1
+ . cri-lock-repos
++ . cri-common
+++ . cri-getconfig
+++ umask 002
+++ getconfig Repos
+++ perl -e '

                use Osstest;
                readglobalconfig();
                print $c{"Repos"} or die $!;
        '

++ repos=/export/home/ianc/repos
++ repos_lock=/export/home/ianc/repos/lock
++ '[' x '!=' x/export/home/ianc/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/export/home/ianc/repos/lock
++ exec with-lock-ex -w /export/home/ianc/repos/lock ./ap-push osstest 2d675fb0de966bae8046b936fe0afe574c6cd9a1
+ branch=osstest
+ revision=2d675fb0de966bae8046b936fe0afe574c6cd9a1
+ . cri-lock-repos
++ . cri-common
+++ . cri-getconfig
+++ umask 002
+++ getconfig Repos
+++ perl -e '

                use Osstest;
                readglobalconfig();
                print $c{"Repos"} or die $!;
        '

++ repos=/export/home/ianc/repos
++ repos_lock=/export/home/ianc/repos/lock
++ '[' x/export/home/ianc/repos/lock '!=' x/export/home/ianc/repos/lock ']'
+ . cri-common
++ . cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=osstest
+ xenbranch=xen-unstable
+ '[' xosstest = xlinux ']'
+ linuxbranch=
+ '[' x = x ']'
+ qemuubranch=qemu-upstream-unstable
+ : tested/2.6.39.x
+ . ap-common
++ : [email protected]
++ : /export/home/ianc/testing.git remote-pretest
++ : git://xenbits.xen.org/xen.git
++ : [email protected]:/home/xen/git/xen.git
++ : git://xenbits.xen.org/staging/qemu-xen-unstable.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/libvirt.git
++ : [email protected]:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : [email protected]:/home/xen/git/rumpuser-xen.git
+++ besteffort_repo https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src +++ cached_repo https://github.com/rumpkernel/rumpkernel-netbsd-src '[fetch=try]' +++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src +++ local 'options=[fetch=try]'
++++ getconfig GitCacheProxy
++++ perl -e '

                use Osstest;
                readglobalconfig();
                print $c{"GitCacheProxy"} or die $!;
        '

+++ local cache=git://drall.uk.xensource.com:9419/
+++ '[' xgit://drall.uk.xensource.com:9419/ '!=' x ']'
+++ echo 'git://drall.uk.xensource.com:9419/https://github.com/rumpkernel/rumpkernel-netbsd-src%20[fetch=try]' ++ : 'git://drall.uk.xensource.com:9419/https://github.com/rumpkernel/rumpkernel-netbsd-src%20[fetch=try]' ++ : git
++ : git://git.seabios.org/seabios.git
++ : [email protected]:/home/xen/git/osstest/seabios.git
++ : git://xenbits.xen.org/osstest/seabios.git
++ : https://github.com/tianocore/edk2.git
++ : [email protected]:/home/xen/git/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/ovmf.git
++ : git://xenbits.xen.org/osstest/linux-firmware.git
++ : [email protected]:/home/osstest/ext/linux-firmware.git
++ : git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
++ : [email protected]:/home/xen/git/linux-pvops.git
++ : git://xenbits.xen.org/linux-pvops.git
++ : tested/linux-3.14
++ : tested/linux-arm-xen
++ '[' xgit://xenbits.xen.org/linux-pvops.git = x ']'
++ '[' x = x ']'
++ : git://xenbits.xen.org/linux-pvops.git
++ : tested/linux-arm-xen
++ : git://git.kernel.org/pub/scm/linux/kernel/git/konrad/xen.git
++ : tested/2.6.39.x
++ : daily-cron.osstest
++ : daily-cron.osstest
++ : daily-cron.osstest
++ : daily-cron.osstest
++ : daily-cron.osstest
++ : daily-cron.osstest
++ : http://hg.uk.xensource.com/carbon/trunk/linux-2.6.27
++ : git://xenbits.xen.org/staging/qemu-upstream-unstable.git
++ : daily-cron.osstest
++ : git://git.qemu.org/qemu.git
++ : git://xenbits.xen.org/osstest/qemu.git
++ : [email protected]:/home/xen/git/osstest/qemu.git
++ : daily-cron.osstest
+ [email protected]:/home/xen/git/linux-pvops.git
+ [email protected]:/home/xen/git/osstest/qemu.git
+ [email protected]:/home/xen/git/qemu-upstream-unstable.git
+ [email protected]:/home/xen/git/xen.git
+ [email protected]:/home/xen/git/libvirt.git
+ [email protected]:/home/xen/git/rumpuser-xen.git
+ [email protected]:/home/xen/git/osstest/seabios.git
+ [email protected]:/home/xen/git/osstest/ovmf.git
+ info_linux_tree osstest
+ case $1 in
+ return 1
+ case "$branch" in
+ git push /export/home/ianc/testing.git 2d675fb0de966bae8046b936fe0afe574c6cd9a1:production To /export/home/ianc/testing.git
a623769..2d675fb 2d675fb0de966bae8046b936fe0afe574c6cd9a1 -> production
+ '[' 'x/export/home/ianc/testing.git remote-pretest' '!=' x ']'
+ :

_______________________________________________
Xen-devel mailing list
[email protected]
http://lists.xen.org/xen-devel

In particular this new $HOME/.xen-osstest/cri-args-hostslists.settings can contain things like "export OSSTEST_CONFIG=production-config-cambridge" to tailor things for a particular instance of osstest running in production mode.

---
 cri-args-hostlists | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

index 04cd6be..bf847f8 100644
@@ -16,7 +16,9 @@
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-
+if [ -e $HOME/.xen-osstest/cri-args-hostslists.settings ]; then
+     source $HOME/.xen-osstest/cri-args-hostslists.settings
+fi
 export OSSTEST_CONFIG=${OSSTEST_CONFIG:-production-config}
 
 check_stop_core () {

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 1/5] cri-args-hostslists: allow instance specific settings"):

...

Makes sense. I'd be happy to do that straightaway if you would prefer.

Ian Campbell writes ("Re: [PATCH OSSTEST v2 1/5] cri-args-hostslists: allow instance specific settings"):

I don't mind. Feel free to make a 6/5 which does this, or fold it in, or whatever.

Ian.

We want to arrange that the master XenProject instance continues to test its own pretest branch while any downstream instances will pickup changes from the master instance's production (i.e. tested) branch, which is published at git://xenbits.xen.org/osstest.git#master.

Add a new configuration option OsstestUpstream which by default is "git://xenbits.xen.org/osstest.git master" and which is cleared to nothing on the master instance via production-config.

If the option is not set then the existing behaviour is unchanged.

If the option is set then osstest branch flights proceed by merging, the referenced branch into the local pretest branch and testing the result.

This must be done:
- in a clone not in the main testing.git in order to avoid inserting merge conflict markers into the active set of scripts.
- in a non-bare repo because git merge requires it.

$repos/osstest is a bare repo which we want to keep that way because using repo_tree_rev_fetch_git to fetch the remote branch is convenient.

So we use $repos/osstest-merge as a temporary merge repo and reclone from the active local repo each time.

All of this happens in ap-fetch version which also fetches the result into FETCH_HEAD into the active repo (the one running the test), such that the existing hard reset in cr-daily-branch can find the commit.

If the merge fails then manual intervention (i.e. a manual merge and push to the _local_ pretest) will be required.

In ap-push we stop pushing to xenbits#master except for the master instance if an upstream is defined. At some point it might be useful to add a configuration option for where to push to but I don't have that requirement right now.

ap-fetch-version-old requires no changes.

---
v2:
- Arrange for $OSSTEST_USE_HEAD=y to take precendence
- drop LOCALREV (which was wrong anyway) in favour of inline branchname
- Rename OSSTEST_REVISION_MERGE as revision_merge to avoid implying it can be set and will be honoured.
- Git in Debian Squeeze lacks -C and --no-edit, adjust accordingly.

---
 Osstest.pm        |  2 ++
 ap-common         |  2 ++
 ap-fetch-version  | 25 ++++++++++++++++++++++---
 ap-push           |  8 +++++++-
 production-config |  2 ++
 5 files changed, 35 insertions(+), 4 deletions(-)

index 7989129..7f72bc6 100644
@@ -102,6 +102,8 @@ sub readglobalconfig () {
     return if $readglobalconfig_done;
     $readglobalconfig_done=1;
 
+    $c{OsstestUpstream} = 'git://xenbits.xen.org/osstest.git master';
+
     $c{HostProp_DhcpWatchMethod} = 'leases dhcp3 infra:5556';
     $c{AuthorizedKeysFiles} = '';
     $c{AuthorizedKeysAppend} = '';

index 64749e3..de7bb1d 100644
@@ -21,6 +21,8 @@
 
 : ${XENBITS:[email protected]}
 
+: ${TREEBRANCH_OSSTEST_UPSTREAM=`getconfig OsstestUpstream`}
+
 : ${TREE_XEN:=git://xenbits.xen.org/xen.git}
 : ${PUSH_TREE_XEN:=$XENBITS:/home/xen/git/xen.git}
 

index 33aaf00..ee502c6 100755
@@ -90,11 +90,30 @@ ovmf)
 		$TREE_OVMF_UPSTREAM master $LOCALREV_OVMF
 	;;
 osstest)
-	if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
+	if [ "x$OSSTEST_USE_HEAD" = "xy" ] ; then
+	    git rev-parse HEAD^0
+	elif [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
+	    revision_merge=`repo_tree_rev_fetch_git osstest \
+		$TREEBRANCH_OSSTEST_UPSTREAM daily-cron.$branch`
+	    echo >&2 "$TREEBRANCH_OSSTEST_UPSTREAM = $revision_merge"
+
+	    rm -rf $repos/osstest-merge >&2
+	    git clone -b pretest $HOME/testing.git $repos/osstest-merge >&2
+
+	    (
+	    cd $repos/osstest-merge
+	    git fetch $repos/osstest daily-cron.$branch:ap-merge >&2
+	    git merge \
+		-m "Automerge of $TREEBRANCH_OSSTEST_UPSTREAM into pretest" \
+		ap-merge >&2
+            )
+
+	    # cr-daily-branch will reset the local tree to this version.
+	    git fetch $repos/osstest-merge pretest
+	    git	rev-parse FETCH_HEAD^0
+	else
 	    git fetch $HOME/testing.git pretest:ap-fetch >&2
             git rev-parse ap-fetch^0
-	else
-	    git rev-parse HEAD^0
 	fi
         ;;
 *)

index c141464..5967b42 100755
@@ -99,7 +99,13 @@ ovmf)
 	;;
 osstest)
 	git push $HOME/testing.git $revision:production
-	git push $XENBITS:/home/xen/git/osstest.git $revision:master
+	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
+	    # could push to instance specific location, but
+	    # certainly not to master instance's xenbits repo!
+	    :
+	else
+	    git push $XENBITS:/home/xen/git/osstest.git $revision:master
+	fi
 	;;
 *)
 	echo >&2 "branch $branch ?"

index 8f20c27..84a3bb7 100644
@@ -15,6 +15,8 @@
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+# This is the master instance, therefore it has no upstream
+OsstestUpstream ''
 
 JobDB Executive
 HostDB Executive

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 2/5] Handle osstest's own local push gate in non-master production instances"):

This turned out to not be quite right in practice. Since pretest is not updated, in the absence of anything actually being changed on each iteration, git would create a new merge, identical to the last time around but for the date, test that and push the result to production. Creating lots of little branches branches in the history. Not what we want at all.

After some discussion IRL and drawing some pictures on a whiteboard here is a replacement version.

I'm about to kick off an adhoc test on the Cambridge VM, but thought I'd best send this out while it was fresh in our heads...

Ian.

From: Ian Campbell <[email protected]>
Date: Wed, 1 Apr 2015 13:50:50 +0100
Subject: [PATCH] Handle osstest's own local push gate in non-master production
instances

We want to arrange that the master XenProject instance continues to test its own pretest branch while any downstream instances will pickup changes from the master instance's production (i.e. tested) branch, which is published at git://xenbits.xen.org/osstest.git#master. We want to also be able to use local pretest for local changes (which may or may not get merged back upstream).

Add a new configuration option OsstestUpstream which by default is "git://xenbits.xen.org/osstest.git master" and which is cleared to nothing on the master instance via production-config.

If the option is not set then the existing behaviour is unchanged.

If the option is set then osstest branch flights will still prefer to test the local pretest branch, but if nothing is pending there then it will proceed by merging the upstream branch into the local production branch and testing the result.

This merge must be done:
- in a clone not in the main testing.git in order to avoid inserting merge conflict markers into the active set of scripts.
- in a non-bare repo because git merge requires it.

$repos/osstest is a bare repo which we want to keep that way because using repo_tree_rev_fetch_git to fetch the remote branch is convenient.

So we use $repos/osstest-merge as a temporary merge repo and reclone from the active local repo each time.

All of this happens in ap-fetch-version.

As part of this arrange that the result is always left in the ap-fetch branch of the for-osstest.git repo (even for existing cases) and the sha1 is produced as output. Resetting to that revision is handled by cr-daily-branch.

If the merge fails then manual intervention (i.e. a manual merge and push to the _local_ pretest) will be required. Likewise if local pretest and local production have diverged manual intervention will be required.

In ap-push we stop pushing to xenbits#master except for the master instance if an upstream is defined. At some point it might be useful to add a configuration option for where to push to but I don't have that requirement right now.

ap-fetch-version-old requires no changes.

---
v3:
- Only merge from upstream if there is nothing pending locally.
- Always update ap-fetch.

v2:
- Arrange for $OSSTEST_USE_HEAD=y to take precendence
- drop LOCALREV (which was wrong anyway) in favour of inline branchname
- Rename OSSTEST_REVISION_MERGE as revision_merge to avoid implying it can be set and will be honoured.
- Git in Debian Squeeze lacks -C and --no-edit, adjust accordingly.

---
 Osstest.pm        |  2 ++
 ap-common         |  2 ++
 ap-fetch-version  | 73 ++++++++++++++++++++++++++++++++++++++++++++++++++++---
 ap-push           |  8 +++++-
 production-config |  2 ++
 5 files changed, 82 insertions(+), 5 deletions(-)

index 7989129..7f72bc6 100644
@@ -102,6 +102,8 @@ sub readglobalconfig () {
     return if $readglobalconfig_done;
     $readglobalconfig_done=1;
 
+    $c{OsstestUpstream} = 'git://xenbits.xen.org/osstest.git master';
+
     $c{HostProp_DhcpWatchMethod} = 'leases dhcp3 infra:5556';
     $c{AuthorizedKeysFiles} = '';
     $c{AuthorizedKeysAppend} = '';

index 64749e3..de7bb1d 100644
@@ -21,6 +21,8 @@
 
 : ${XENBITS:[email protected]}
 
+: ${TREEBRANCH_OSSTEST_UPSTREAM=`getconfig OsstestUpstream`}
+
 : ${TREE_XEN:=git://xenbits.xen.org/xen.git}
 : ${PUSH_TREE_XEN:=$XENBITS:/home/xen/git/xen.git}
 

index c141464..5967b42 100755
@@ -99,7 +99,13 @@ ovmf)
 	;;
 osstest)
 	git push $HOME/testing.git $revision:production
-	git push $XENBITS:/home/xen/git/osstest.git $revision:master
+	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
+	    # could push to instance specific location, but
+	    # certainly not to master instance's xenbits repo!
+	    :
+	else
+	    git push $XENBITS:/home/xen/git/osstest.git $revision:master
+	fi
 	;;
 *)
 	echo >&2 "branch $branch ?"

index 8f20c27..84a3bb7 100644
@@ -15,6 +15,8 @@
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+# This is the master instance, therefore it has no upstream
+OsstestUpstream ''
 
 JobDB Executive
 HostDB Executive

2.1.4

Except the filer seems to have just died for the second time today. I suspect I'll be putting a hold on this work until we can switch over to reliable infrastructure (which is in progress AIUI)

Ian.

Right now all it does is keep osstest itself up to date.

---
 crontab-cambridge | 5 +++++
 1 file changed, 5 insertions(+)
 create mode 100644 crontab-cambridge

new file mode 100644
index 0000000..ef0d234
@@ -0,0 +1,5 @@
+PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
[email protected],[email protected]
+# m		h	dom mon dow	command
+49		1	* * *		cd testing.git && BRANCHES=osstest		./cr-for-branches branches -w "./cr-daily-branch --real"
+3		4	* * *		savelog -c28 testing.git/tmp/cr-for-branches.log >/dev/null

2.1.4

By arranging for cr-publish-flight-logs to ignore --push-harness if either of HarnessPublishGitRepoDir or HarnessPublishGitUserHost are not specified

---
v2:
- Avoid logm which isn't available here, wasn't saying much of use anyway.
- Syntax fix (is not a function, so exit not return)

Perhaps we should have our own tree for such things, but for now just nobble it.

---
 cr-publish-flight-logs      | 2 ++
 production-config-cambridge | 3 ---
 2 files changed, 2 insertions(+), 3 deletions(-)

index 1feb52d..2e96f60 100755
@@ -43,6 +43,8 @@ if ($push_harness) {
     my $githost= $c{HarnessPublishGitUserHost};
     my $gitdir= $c{HarnessPublishGitRepoDir};
 
+    exit 0 if (!$githost || !$gitdir);
+
     system_checked("git push $githost:$gitdir HEAD:refs/heads/flight-$flight");
     system_checked("ssh $githost 'cd $gitdir && git update-server-info'");
 }

index 1404d10..e44f3b0 100644
@@ -50,9 +50,6 @@ GlobalLockDir /export/home/osstest/testing.git
 LogsPublish= "$c{Publish}/logs"
 ResultsPublish= "$c{Publish}/results"
 
-HarnessPublishGitUserHost [email protected]
-HarnessPublishGitRepoDir git/osstest.git
-
 ReportTrailer= <<END
 Logs, config files, etc. are available at
     $c{ReportHtmlPubBaseUrl}

2.1.4

http://osstest.cam.xci-test.com/~osstest/testlogs already exists and points to the live logs directory, so switch PubBaseUrl to that in the Cambridge config such that email reports etc contain it. This won't be externally accessible but I think that won't matter now that the master production instance is elsewhere.

Arrange that cr-publish-flight-logs doesn't publish the corresponding thing if either LogsPublish or ResultsPublish is not set, and unset them in the Cambridge config.

Likewise arrange that cr-ensure-disk-space doesn't do anything if the configuration variable passed as an option is not set, and unset Publish (the base for {Logs,Results}Publish) in the Cambridge config.

---
v2: Check the config variable and not its name.

---
 cr-ensure-disk-space        | 2 ++
 cr-publish-flight-logs      | 4 ++--
 production-config-cambridge | 6 +-----
 3 files changed, 5 insertions(+), 7 deletions(-)

index 0ee84c7..3788e63 100755
@@ -42,6 +42,8 @@ our ($cfgbase) = @ARGV;
 
 csreadconfig();
 
+exit 0 unless $c{$cfgbase};
+
 open LOCK, "> $c{GlobalLockDir}/publish-lock" or die $!;
 flock LOCK, LOCK_EX or die $!;
 

index 2e96f60..94c74c8 100755
@@ -59,5 +59,5 @@ sub copydir ($$) {
     $!=0; $?=0; system @cmd; die "rsync $? $!" if $? or $!;
 }
 
-copydir("$c{Logs}/$flight/", "$c{LogsPublish}/$flight");
-copydir("$c{Results}/", "$c{ResultsPublish}");
+copydir("$c{Logs}/$flight/", "$c{LogsPublish}/$flight") if $c{LogsPublish};
+copydir("$c{Results}/", "$c{ResultsPublish}") if $c{ResultsPublish};

index e44f3b0..a144cef 100644
@@ -40,16 +40,12 @@ TestHostKeypairPath /export/home/osstest/.ssh/id_rsa_osstest
 
 GitCacheProxy git://drall.uk.xensource.com:9419/
 
-PubBaseUrl http://www.chiark.greenend.org.uk/~xensrcts
+PubBaseUrl http://osstest.cam.xci-test.com/~osstest/testlogs
 ReportHtmlPubBaseUrl="$c{PubBaseUrl}/logs"
 ResultsHtmlPubBaseUrl="$c{PubBaseUrl}/results"
 
-Publish [email protected]:/home/ian/work/xc_osstest
 GlobalLockDir /export/home/osstest/testing.git
 
-LogsPublish= "$c{Publish}/logs"
-ResultsPublish= "$c{Publish}/results"
-
 ReportTrailer= <<END
 Logs, config files, etc. are available at
     $c{ReportHtmlPubBaseUrl}

2.1.4

Ian Campbell writes ("[PATCH OSSTEST v2 5/5] cambridge: Stop publishing logs to chiark"):

Ian Campbell writes ("[PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances"):

I think you are probably right, since this particular series contains the merging-from-an-upsream machinery.

Ian.

Since you have now acked everything may I push to the Cambridge branch (once the VM is working again)?

(modulo if you want me to make the minor change discussed on 1/5 now or not)

Ian.

Ian Campbell writes ("Re: [PATCH OSSTEST v2 0/5] Tweaks to allow running non-master production instances"):

Yes.

Either way is fine.

Thanks,
Ian.

I created the file on both sites, but I didn't change this patch since we are pushing this to Cambridge first if it does cause an issue when merged back into mainline it'll be annoying to cleanup, better to let a future change go through the usual mainline gate alone.

I've just pushed the following, which is a bunch of stuff from upstream osstest and these five patches, to Cambridge pretest and then:

$ rm branches/for-osstest.git/stop 

Lets see if the VM stays up long enough to test them!

Ian.

$ git log --oneline a623769..41ac504
41ac504 cambridge: Stop publishing logs to chiark
bcc9c68 cambridge: Do not try to push harness to XenProject instance output
6196543 cambridge: Add a crontab
7007b0a Handle osstest's own local push gate in non-master production instances
5926203 cri-args-hostslists: allow instance specific settings
eff0c77 cambridge: Set HostProp_DhcpWatchMethod back to woking
8f1dc3f ts-host-ping-check: Do not run if host is being reused
ba691ef standalone: Add --dry-run option for run-job.
effe70a tcl: Handle environment variables which are unset.
4a938b2 Merge branch 'production' of osstest.test-lab:/home/osstest/testing into wip
2d1aa25 production-config: Adjust LogsMinSpaceMby to not be half of the VM's disk
5ae027f Merge branch 'colo.2015-03-31' of xenbits.xen.org:/home/iwj/ext/osstest into queue
4bf33f8 production-config*: Set HostnameSortSwapWords as appropriate
e2008dc (Re)Add production config for Cambridge instance
f89d6ac Osstest.pm: Add "use IO::File"
0ab790c ts-host-install: honour arch-specific boot append properties
e654ecc ts-host-ping-check: New ubiquitous test step
a80d215 sg-report-flight: Produce better output for running jobs.
6ec412e sg-report-flight: Sort email output by results, not job name
6cbaf65 cs-adjust-flight: Support negation of foo-specs
fdddba7 config: turn off HostnameSortSwapWords by default
ea335bc start/stop the guest 10 times in the standard test jobs
4d6bbe0 README.dev: Document steps after restarting ms-ownerdaemon
97fa466 uboot: make flask loading address host property
ab441f3 uboot: do not use readlink on xsm policy
dce7456 Merge remote-tracking branch 'origin/incoming' into wip.colo
bf3a468 Remove passthrough tests
7d87cf0 Empty commit to force test run
bc50a13 Configs: Replace xenbits.xensource.com with xenbits.xen.org
da7c22d mg-branch-setup: Make initial push optional in mg-branch-setup
831c94e ts-debian-install: Honour DebianMirrorProxy, HttpProxy
a141aca production-config: Configuration for new production colo
310c5b5 Branches: Rename "incoming" branch to "production"
b294e14 config defaults: Do not use woking for dhcp leases
2fedf11 ts-kernel-build: enable CONFIG_SCSI_HPSA
347eadf ts-kernel-build: enable CONFIG_SCSI_ISCI
591ed47 ts-kernel-build: Document a hacky way to get CONFIG_... options
4d0dba4 HostnameSortSwapWords: Make name order mangling configurable
a4debc0 mg-execute-flight: New -P (publish) option
41e9bf0 cr-ensure-disk-space: Honour different config for published log expiry
2ad7b3f cr-ensure-disk-space: Operate locally entirely separately.
2563b91 cr-ensure-disk-space: Permit argument to specify local directory.
dd6be0f cr-ensure-disk-space: Fix -n option
43a0974 cr-ensure-disk-space: Take argument for what to clean
1d25a8c cr-ensure-disk-space: Do listing of flights on target.
d6edb5d cr-ensure-disk-space: Do df check on target.
438637a cr-ensure-disk-space: Break out ontarget
fdc2659 cr-ensure-disk-space: Move LogsPublish decomposition up
d8b13ac cr-ensure-disk-space: Emacs perl-mode syntax workaround
f16e621 daemons: Allow QueueDaemon and OwnerDaemon to run on different hosts
0ff3bb1 target_cmd_build: Honour $c{HttpProxy}
a11fc63 HostDB Executive: Option to suppress "configdb"
2b5aa29 readglobalconfig: Support Include
f5b354a readglobalconfig: Break out $readcfg as a sub
bf2b666 Debian mirrors: Support proxy, and default to ftp.debian.org if specified
eb805e8 pxe setup: mg-hosts mkpxedir has -n (dry run) option
a865cc2 pxe setup: New TftpPxeTemplatesReal feature
14e0e8d Osstest: Default config: Use foreach for empty defaults
bbcce4d mg-hosts: Provide new "power" subcommand
68f35a1 pxe setup: Define %name% in templates
e8a5b82 PDU: pdu-msw: Support APC v6 firmware
9705c37 PDU: pdu-msw: Split $read_oid and $write_oid
0fb0a00 PDU: msw: support arbitrary arguments

All sites now have a suitable $HOME/.xen-osstest/cri-args-hostslists.settings in place which does this.

---
To be applied once the following commit passes Cambridge push gate, is merged into the master instance and passes the push gate there.

commit 5926203d0851792e4f6ce26df5a48ba5c79456e1

Author: Ian Campbell <[email protected]>
Date:   Wed Apr 15 11:04:53 2015 +0100

cri-args-hostslists: allow instance specific settings

In particular this new $HOME/.xen-osstest/cri-args-hostslists.settings can contain things like "export OSSTEST_CONFIG=production-config-cambridge" to tailor things for a particular instance of osstest running in production mode.

---
 cri-args-hostlists | 1 -
 1 file changed, 1 deletion(-)

index bf847f8..6aa6927 100644
@@ -19,7 +19,6 @@
 if [ -e $HOME/.xen-osstest/cri-args-hostslists.settings ]; then
      source $HOME/.xen-osstest/cri-args-hostslists.settings
 fi
-export OSSTEST_CONFIG=${OSSTEST_CONFIG:-production-config}
 
 check_stop_core () {
 	if [ "x$OSSTEST_IGNORE_STOP" = xy ]; then return; fi

2.1.4