Skip to contentSkip to conversation

Tweaks to allow running non-master production instances

PATCH Superseded Ian Campbell 2 participants 14 messages Apr 1, 2015 – Apr 2, 2015

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover Tweaks to allow running non-master production instances

Now that the Cambridge osstest instance is no longer the XenProject master instance it needs some tweaks to allow us to keep using it for local testing.

Ian.

#1 lore raw

PATCH 1/5 cri-args-hostslists: allow instance specific settings

In particular this new $HOME/.xen-osstest/cri-args-hostslists.settings can contain things like "export OSSTEST_CONFIG=production-config-cambridge" to tailor things for a particular instance of osstest running in production mode.

---
 cri-args-hostlists | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
cri-args-hostlists +3 -1
index 04cd6be..bf847f8 100644
@@ -16,7 +16,9 @@
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-
+if [ -e $HOME/.xen-osstest/cri-args-hostslists.settings ]; then
+     source $HOME/.xen-osstest/cri-args-hostslists.settings
+fi
 export OSSTEST_CONFIG=${OSSTEST_CONFIG:-production-config}
 
 check_stop_core () {
Signature · 1 line
2.1.4
Acked-by on v1 1/5
#2 lore raw

Re: [PATCH OSSTEST 1/5] cri-args-hostslists: allow instance specific settings

Ian Campbell writes ("[PATCH OSSTEST 1/5] cri-args-hostslists: allow instance specific settings"):

Quoted 4 lines

In particular this new $HOME/.xen-osstest/cri-args-hostslists.settings can contain things like "export OSSTEST_CONFIG=production-config-cambridge" to tailor things for a particular instance of osstest running in production mode.

#3 lore raw

PATCH 2/5 Handle osstest's own local push gate in non-master production instances

In reply to Ian Campbell

We want to arrange that the master XenProject instance continues to test its own pretest branch while any downstream instances will pickup changes from the master instance's production (i.e. tested) branch, which is published at git://xenbits.xen.org/osstest.git#master.

Add a new configuration option OsstestUpstream which by default is "git://xenbits.xen.org/osstest.git master" and which is cleared to nothing on the master instance via production-config.

If the option is not set then the existing behaviour is unchanged.

If the option is set then osstest branch flights proceed by merging, the referenced branch into the local pretest branch and testing the result.

This must be done:
- in a clone not in the main testing.git in order to avoid inserting merge conflict markers into the active set of scripts.
- in a non-bare repo because git merge requires it.

$repos/osstest is a bare repo which we want to keep that way because using repo_tree_rev_fetch_git to fetch the remote branch is convenient.

So we use $repos/osstest-merge as a temporary merge repo and reclone from the active local repo each time.

All of this happens in ap-fetch version which also fetches the result into FETCH_HEAD into the active repo (the one running the test), such that the existing hard reset in cr-daily-branch can find the commit.

If the merge fails then manual intervention (i.e. a manual merge and push to the _local_ pretest) will be required.

In ap-push we stop pushing to xenbits#master except for the master instance if an upstream is defined. At some point it might be useful to add a configuration option for where to push to but I don't have that requirement right now.

ap-fetch-version-old requires no changes.

---
 Osstest.pm        |  2 ++
 ap-common         |  3 +++
 ap-fetch-version  | 19 ++++++++++++++++++-
 ap-push           |  8 +++++++-
 production-config |  2 ++
 5 files changed, 32 insertions(+), 2 deletions(-)
Osstest.pm +2 -0
index 7989129..7f72bc6 100644
@@ -102,6 +102,8 @@ sub readglobalconfig () {
     return if $readglobalconfig_done;
     $readglobalconfig_done=1;
 
+    $c{OsstestUpstream} = 'git://xenbits.xen.org/osstest.git master';
+
     $c{HostProp_DhcpWatchMethod} = 'leases dhcp3 infra:5556';
     $c{AuthorizedKeysFiles} = '';
     $c{AuthorizedKeysAppend} = '';
ap-common +3 -0
index 64749e3..fadeb2b 100644
@@ -21,6 +21,8 @@
 
 : ${XENBITS:[email protected]}
 
+: ${TREEBRANCH_OSSTEST_UPSTREAM=`getconfig OsstestUpstream`}
+
 : ${TREE_XEN:=git://xenbits.xen.org/xen.git}
 : ${PUSH_TREE_XEN:=$XENBITS:/home/xen/git/xen.git}
 
@@ -76,6 +78,7 @@ fi
 : ${TREE_LINUX2639:=${KERNEL_SCM}/konrad/xen.git}
 : ${TAG_LINUX2639:=stable/2.6.39.x}
 
+: ${LOCALREV_OSSTEST:=daily-cron.$branch}
 : ${LOCALREV_XEN:=daily-cron.$branch}
 : ${LOCALREV_LINUX:=daily-cron.$branch}
 : ${LOCALREV_LIBVIRT:=daily-cron.$branch}
ap-fetch-version +18 -1
index 33aaf00..efd3139 100755
@@ -90,7 +90,24 @@ ovmf)
 		$TREE_OVMF_UPSTREAM master $LOCALREV_OVMF
 	;;
 osstest)
-	if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
+	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
+	    OSSTEST_REVISION_MERGE=`repo_tree_rev_fetch_git osstest \
+		$TREEBRANCH_OSSTEST_UPSTREAM $LOCALREV_OVMF`
+	    echo >&2 "$TREEBRANCH_OSSTEST_UPSTREAM = $OSSTEST_REVISION_MERGE"
+
+	    rm -rf $repos/osstest-merge >&2
+	    git clone -b pretest $HOME/testing.git $repos/osstest-merge >&2
+
+	    git -C $repos/osstest-merge \
+		fetch $repos/osstest $LOCALREV_OVMF:ap-merge >&2
+	    git -C $repos/osstest-merge merge --no-edit \
+		-m "Automerge of $TREEBRANCH_OSSTEST_UPSTREAM into pretest" \
+		ap-merge >&2
+
+	    # cr-daily-branch will reset the local tree to this version.
+	    git fetch $repos/osstest-merge pretest
+	    git	rev-parse FETCH_HEAD^0
+	elif [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
 	    git fetch $HOME/testing.git pretest:ap-fetch >&2
             git rev-parse ap-fetch^0
 	else
ap-push +7 -1
index c141464..5967b42 100755
@@ -99,7 +99,13 @@ ovmf)
 	;;
 osstest)
 	git push $HOME/testing.git $revision:production
-	git push $XENBITS:/home/xen/git/osstest.git $revision:master
+	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
+	    # could push to instance specific location, but
+	    # certainly not to master instance's xenbits repo!
+	    :
+	else
+	    git push $XENBITS:/home/xen/git/osstest.git $revision:master
+	fi
 	;;
 *)
 	echo >&2 "branch $branch ?"
production-config +2 -0
index 2ffefdd..9983025 100644
@@ -15,6 +15,8 @@
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
+# This is the master instance, therefore it has no upstream
+OsstestUpstream ''
 
 JobDB Executive
 HostDB Executive
Signature · 1 line
2.1.4
#4 lore raw

Re: [PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances

Ian Campbell writes ("[PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances"):

Quoted 4 lines

We want to arrange that the master XenProject instance continues to test its own pretest branch while any downstream instances will pickup changes from the master instance's production (i.e. tested) branch, which is published at git://xenbits.xen.org/osstest.git#master.

... ;;

ap-fetch-version ovmf) · line 93

Quoted 3 lines

osstest)

ap-fetch-version +1 -1
-	if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
+	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then

Shouldn't the OSSTEST_USE_HEAD option override this ? Ie I think this is the wrong way round.

ap-fetch-version ovmf) · line 95

Quoted 2 lines
ap-fetch-version +2 -0
+	    OSSTEST_REVISION_MERGE=`repo_tree_rev_fetch_git osstest \
+		$TREEBRANCH_OSSTEST_UPSTREAM $LOCALREV_OVMF`

OSSTEST_REVISION_MERGE doesn't seem to be an environment variable. I think it should either be honoured if pre-set, or just be an ordinary (lowercase) variable name (in which case it doesn't need the OSSTEST prefix).

ap-fetch-version ovmf) · line 102

Quoted 7 lines
ap-fetch-version +7 -0
+	    echo >&2 "$TREEBRANCH_OSSTEST_UPSTREAM = $OSSTEST_REVISION_MERGE"
+
+	    rm -rf $repos/osstest-merge >&2
+	    git clone -b pretest $HOME/testing.git $repos/osstest-merge >&2
+
+	    git -C $repos/osstest-merge \
+		fetch $repos/osstest $LOCALREV_OVMF:ap-merge >&2

LOCALREV_OVMF (here and earlier). It's not clear to me that you actually need to indirect it; you could just specify a branch name.

The rest looks fine.

Having thought about how to override things, particularly:

ap-push ovmf) · line 106

Quoted 5 lines
ap-push +5 -0
+	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
+	    # could push to instance specific location, but
+	    # certainly not to master instance's xenbits repo!
+	    :
+	else

I think ap-fetch-* and ap-push should have a general purpose hook which occurs just after ap-common is loaded, something like this:

  diff --git a/ap-fetch-version b/ap-fetch-version
  index 33aaf00..a3028f6 100755
  --- a/ap-fetch-version
  +++ b/ap-fetch-version
  @@ -31,6 +31,8 @@ if info_linux_tree "$branch"; then
          exit 0
   fi

  +. ${OSSTEST_HOOK_AP_FETCH-/dev/null}
  +
   case "$branch" in
   xen-3.*)
          ./sg-hg-heads sh -ec '

We can postpone actually writing that until someone needs it.

Ian.

#5 lore raw

Re: [PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances

Quoted 5 lines · Ian Jackson

Ian Campbell writes ("[PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances"):

Quoted 4 lines

We want to arrange that the master XenProject instance continues to test its own pretest branch while any downstream instances will pickup changes from the master instance's production (i.e. tested) branch, which is published at git://xenbits.xen.org/osstest.git#master.

Quoted 7 lines

... ;;

ap-fetch-version ovmf) · line 93

Quoted 3 lines

osstest)

ap-fetch-version +1 -1
-	if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then
+	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then

Shouldn't the OSSTEST_USE_HEAD option override this ? Ie I think this is the wrong way round.

Yes, I think you are right.

I think I just ended up with this because I was testing via the standalone wrapper which sets USE_HEAD, but I'll test another way.

Quoted 7 lines

ap-fetch-version ovmf) · line 95

Quoted 2 lines
ap-fetch-version +2 -0
+	    OSSTEST_REVISION_MERGE=`repo_tree_rev_fetch_git osstest \
+		$TREEBRANCH_OSSTEST_UPSTREAM $LOCALREV_OVMF`

OSSTEST_REVISION_MERGE doesn't seem to be an environment variable. I think it should either be honoured if pre-set, or just be an ordinary (lowercase) variable name (in which case it doesn't need the OSSTEST prefix).

OK. Any preference as to which?

As it stands the only reason for the variable is to sink the output from repo_tree_rev_fetch and log it.

Quoted 11 lines

ap-fetch-version ovmf) · line 102

Quoted 8 lines
ap-fetch-version +7 -0
+	    echo >&2 "$TREEBRANCH_OSSTEST_UPSTREAM = $OSSTEST_REVISION_MERGE"
+
+	    rm -rf $repos/osstest-merge >&2
+	    git clone -b pretest $HOME/testing.git $repos/osstest-merge >&2
+
+	    git -C $repos/osstest-merge \
+		fetch $repos/osstest $LOCALREV_OVMF:ap-merge >&2

LOCALREV_OVMF (here and earlier). It's not clear to me that you actually need to indirect it; you could just specify a branch name.

OK.

Quoted 27 lines

The rest looks fine.

Having thought about how to override things, particularly:

ap-push ovmf) · line 106

Quoted 5 lines
ap-push +5 -0
+	if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
+	    # could push to instance specific location, but
+	    # certainly not to master instance's xenbits repo!
+	    :
+	else

I think ap-fetch-* and ap-push should have a general purpose hook which occurs just after ap-common is loaded, something like this:

  diff --git a/ap-fetch-version b/ap-fetch-version
  index 33aaf00..a3028f6 100755
  --- a/ap-fetch-version
  +++ b/ap-fetch-version
  @@ -31,6 +31,8 @@ if info_linux_tree "$branch"; then
          exit 0
   fi

  +. ${OSSTEST_HOOK_AP_FETCH-/dev/null}
  +
   case "$branch" in
   xen-3.*)
          ./sg-hg-heads sh -ec '
Quoted 1 line

We can postpone actually writing that until someone needs it.

Gladly ;-)

Ian.

#6 lore raw

PATCH 3/5 cambridge: Add a crontab

In reply to Ian Campbell

Right now all it does is keep osstest itself up to date.

---
 crontab-cambridge | 5 +++++
 1 file changed, 5 insertions(+)
 create mode 100644 crontab-cambridge
crontab-cambridge new file +5 -0
new file mode 100644
index 0000000..ef0d234
@@ -0,0 +1,5 @@
+PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
[email protected],[email protected]
+# m		h	dom mon dow	command
+49		1	* * *		cd testing.git && BRANCHES=osstest		./cr-for-branches branches -w "./cr-daily-branch --real"
+3		4	* * *		savelog -c28 testing.git/tmp/cr-for-branches.log >/dev/null
Signature · 1 line
2.1.4
Acked-by on v1 3/5
#7 lore raw

Re: [PATCH OSSTEST 3/5] cambridge: Add a crontab

Ian Campbell writes ("[PATCH OSSTEST 3/5] cambridge: Add a crontab"):

Quoted 1 line

Right now all it does is keep osstest itself up to date.

#8 lore raw

PATCH 4/5 cambridge: Do not try to push harness to XenProject instance output

In reply to Ian Campbell

By arranging for cr-publish-flight-logs to ignore --push-harness if either of HarnessPublishGitRepoDir or HarnessPublishGitUserHost are not specified

---
Perhaps we should have our own tree for such things, but for now just nobble it.

---
 cr-publish-flight-logs      | 4 ++++
 production-config-cambridge | 3 ---
 2 files changed, 4 insertions(+), 3 deletions(-)
cr-publish-flight-logs +4 -0
index 1feb52d..5311015 100755
@@ -43,6 +43,10 @@ if ($push_harness) {
     my $githost= $c{HarnessPublishGitUserHost};
     my $gitdir= $c{HarnessPublishGitRepoDir};
 
+    if (!$githost || !$gitdir) {
+        logm("Nowhere to publish harness to");
+        return 0;
+    }
     system_checked("git push $githost:$gitdir HEAD:refs/heads/flight-$flight");
     system_checked("ssh $githost 'cd $gitdir && git update-server-info'");
 }
production-config-cambridge +0 -3
index ad4e44c..043a73c 100644
@@ -50,9 +50,6 @@ GlobalLockDir /export/home/osstest/testing.git
 LogsPublish= "$c{Publish}/logs"
 ResultsPublish= "$c{Publish}/results"
 
-HarnessPublishGitUserHost [email protected]
-HarnessPublishGitRepoDir git/osstest.git
-
 ReportTrailer= <<END
 Logs, config files, etc. are available at
     $c{ReportHtmlPubBaseUrl}
Signature · 1 line
2.1.4
Acked-by on v1 4/5
#9 lore raw

Re: [PATCH OSSTEST 4/5] cambridge: Do not try to push harness to XenProject instance output

Ian Campbell writes ("[PATCH OSSTEST 4/5] cambridge: Do not try to push harness to XenProject instance output"):

Quoted 3 lines

By arranging for cr-publish-flight-logs to ignore --push-harness if either of HarnessPublishGitRepoDir or HarnessPublishGitUserHost are not specified

#10 lore raw

PATCH 5/5 cambridge: Stop publishing logs to chiark

In reply to Ian Campbell

http://osstest.cam.xci-test.com/~osstest/testlogs already exists and points to the live logs directory, so switch PubBaseUrl to that in the Cambridge config such that email reports etc contain it. This won't be externally accessible but I think that won't matter now that the master production instance is elsewhere.

Arrange that cr-publish-flight-logs doesn't publish the corresponding thing if either LogsPublish or ResultsPublish is not set, and unset them in the Cambridge config.

Likewise arrange that cr-ensure-disk-space doesn't do anything if the configuration variable passed as an option is not set, and unset Publish (the base for {Logs,Results}Publish) in the Cambridge config.

---
 cr-ensure-disk-space        | 2 ++
 cr-publish-flight-logs      | 4 ++--
 production-config-cambridge | 6 +-----
 3 files changed, 5 insertions(+), 7 deletions(-)
cr-ensure-disk-space +2 -0
index 0ee84c7..84b1890 100755
@@ -40,6 +40,8 @@ die unless @ARGV==1;
 
 our ($cfgbase) = @ARGV;
 
+exit 0 unless $cfgbase;
+
 csreadconfig();
 
 open LOCK, "> $c{GlobalLockDir}/publish-lock" or die $!;
cr-publish-flight-logs +2 -2
index 5311015..3894351 100755
@@ -61,5 +61,5 @@ sub copydir ($$) {
     $!=0; $?=0; system @cmd; die "rsync $? $!" if $? or $!;
 }
 
-copydir("$c{Logs}/$flight/", "$c{LogsPublish}/$flight");
-copydir("$c{Results}/", "$c{ResultsPublish}");
+copydir("$c{Logs}/$flight/", "$c{LogsPublish}/$flight") if $c{LogsPublish};
+copydir("$c{Results}/", "$c{ResultsPublish}") if $c{ResultsPublish};
production-config-cambridge +1 -5
index 043a73c..b3a7da5 100644
@@ -40,16 +40,12 @@ TestHostKeypairPath /export/home/osstest/.ssh/id_rsa_osstest
 
 GitCacheProxy git://drall.uk.xensource.com:9419/
 
-PubBaseUrl http://www.chiark.greenend.org.uk/~xensrcts
+PubBaseUrl http://osstest.cam.xci-test.com/~osstest/testlogs
 ReportHtmlPubBaseUrl="$c{PubBaseUrl}/logs"
 ResultsHtmlPubBaseUrl="$c{PubBaseUrl}/results"
 
-Publish [email protected]:/home/ian/work/xc_osstest
 GlobalLockDir /export/home/osstest/testing.git
 
-LogsPublish= "$c{Publish}/logs"
-ResultsPublish= "$c{Publish}/results"
-
 ReportTrailer= <<END
 Logs, config files, etc. are available at
     $c{ReportHtmlPubBaseUrl}
Signature · 1 line
2.1.4
#11 lore raw

Re: [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark

Ian Campbell writes ("[PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark"):

Quoted 26 lines

http://osstest.cam.xci-test.com/~osstest/testlogs already exists and points to the live logs directory, so switch PubBaseUrl to that in the Cambridge config such that email reports etc contain it. This won't be externally accessible but I think that won't matter now that the master production instance is elsewhere.

Arrange that cr-publish-flight-logs doesn't publish the corresponding thing if either LogsPublish or ResultsPublish is not set, and unset them in the Cambridge config.

Likewise arrange that cr-ensure-disk-space doesn't do anything if the configuration variable passed as an option is not set, and unset Publish (the base for {Logs,Results}Publish) in the Cambridge config.

---
 cr-ensure-disk-space        | 2 ++
 cr-publish-flight-logs      | 4 ++--
 production-config-cambridge | 6 +-----
 3 files changed, 5 insertions(+), 7 deletions(-)
cr-ensure-disk-space +0 -0
index 0ee84c7..84b1890 100755

cr-ensure-disk-space die unless @ARGV==1; · line 45

Quoted 6 lines
cr-ensure-disk-space +1 -0
@@ -40,6 +40,8 @@ die unless @ARGV==1;
 
 our ($cfgbase) = @ARGV;
 
+exit 0 unless $cfgbase;
 
 csreadconfig();

Don't you mean "unless $c{$cfgbase}" ? And then it has to come after csreadconfig. (Did you test this ?)

The rest LGTM.

Thanks,
Ian.

#12 lore raw

Re: [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark

Quoted 28 lines · Ian Jackson

Ian Campbell writes ("[PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark"):

Quoted 27 lines

http://osstest.cam.xci-test.com/~osstest/testlogs already exists and points to the live logs directory, so switch PubBaseUrl to that in the Cambridge config such that email reports etc contain it. This won't be externally accessible but I think that won't matter now that the master production instance is elsewhere.

Arrange that cr-publish-flight-logs doesn't publish the corresponding thing if either LogsPublish or ResultsPublish is not set, and unset them in the Cambridge config.

Likewise arrange that cr-ensure-disk-space doesn't do anything if the configuration variable passed as an option is not set, and unset Publish (the base for {Logs,Results}Publish) in the Cambridge config.

---
 cr-ensure-disk-space        | 2 ++
 cr-publish-flight-logs      | 4 ++--
 production-config-cambridge | 6 +-----
 3 files changed, 5 insertions(+), 7 deletions(-)
cr-ensure-disk-space +0 -0
index 0ee84c7..84b1890 100755
@@ -40,6 +40,8 @@ die unless @ARGV==1;
 
Quoted 7 lines

cr-ensure-disk-space die unless @ARGV==1;

Quoted 5 lines
cr-ensure-disk-space +1 -0
 our ($cfgbase) = @ARGV;
 
+exit 0 unless $cfgbase;
 
 csreadconfig();

Don't you mean "unless $c{$cfgbase}" ?

I think I do, yes.

Quoted 1 line

And then it has to come after csreadconfig. (Did you test this ?)

I was running in standalone mode, so I don't think this code will have actually been run, I was relying on my ability to reason about the code, which apparently failed :-/.

Shall I run an adhoc flight with this change in it?

Ian.

#13 lore raw

Re: [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark

Quoted 1 line · Ian Campbell

Shall I run an adhoc flight with this change in it?

FYI, I think I've now figured out how to run something approximating a "proper" osstest flight from something other than ~osstest/testing.git via the cronjob, so one is in progress now...

Ian.