Tweaks to allow running non-master production instances
cover Tweaks to allow running non-master production instances
Now that the Cambridge osstest instance is no longer the XenProject master instance it needs some tweaks to allow us to keep using it for local testing.
Ian.
PATCH 1/5 cri-args-hostslists: allow instance specific settings
In particular this new $HOME/.xen-osstest/cri-args-hostslists.settings can contain things like "export OSSTEST_CONFIG=production-config-cambridge" to tailor things for a particular instance of osstest running in production mode.
--- cri-args-hostlists | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
cri-args-hostlists +3 -1
@@ -16,7 +16,9 @@ # You should have received a copy of the GNU Affero General Public License # along with this program. If not, see <http://www.gnu.org/licenses/>. - +if [ -e $HOME/.xen-osstest/cri-args-hostslists.settings ]; then + source $HOME/.xen-osstest/cri-args-hostslists.settings +fi export OSSTEST_CONFIG=${OSSTEST_CONFIG:-production-config} check_stop_core () {
Signature · 1 line
2.1.4
Re: [PATCH OSSTEST 1/5] cri-args-hostslists: allow instance specific settings
Ian Campbell writes ("[PATCH OSSTEST 1/5] cri-args-hostslists: allow instance specific settings"):
Quoted 4 lines
In particular this new $HOME/.xen-osstest/cri-args-hostslists.settings can contain things like "export OSSTEST_CONFIG=production-config-cambridge" to tailor things for a particular instance of osstest running in production mode.
PATCH 2/5 Handle osstest's own local push gate in non-master production instances
In reply to Ian CampbellWe want to arrange that the master XenProject instance continues to test its own pretest branch while any downstream instances will pickup changes from the master instance's production (i.e. tested) branch, which is published at git://xenbits.xen.org/osstest.git#master.
Add a new configuration option OsstestUpstream which by default is "git://xenbits.xen.org/osstest.git master" and which is cleared to nothing on the master instance via production-config.
If the option is not set then the existing behaviour is unchanged.
If the option is set then osstest branch flights proceed by merging, the referenced branch into the local pretest branch and testing the result.
This must be done:
- in a clone not in the main testing.git in order to avoid inserting merge conflict markers into the active set of scripts.
- in a non-bare repo because git merge requires it.
$repos/osstest is a bare repo which we want to keep that way because using repo_tree_rev_fetch_git to fetch the remote branch is convenient.
So we use $repos/osstest-merge as a temporary merge repo and reclone from the active local repo each time.
All of this happens in ap-fetch version which also fetches the result into FETCH_HEAD into the active repo (the one running the test), such that the existing hard reset in cr-daily-branch can find the commit.
If the merge fails then manual intervention (i.e. a manual merge and push to the _local_ pretest) will be required.
In ap-push we stop pushing to xenbits#master except for the master instance if an upstream is defined. At some point it might be useful to add a configuration option for where to push to but I don't have that requirement right now.
ap-fetch-version-old requires no changes.
--- Osstest.pm | 2 ++ ap-common | 3 +++ ap-fetch-version | 19 ++++++++++++++++++- ap-push | 8 +++++++- production-config | 2 ++ 5 files changed, 32 insertions(+), 2 deletions(-)
Osstest.pm +2 -0
@@ -102,6 +102,8 @@ sub readglobalconfig () { return if $readglobalconfig_done; $readglobalconfig_done=1; + $c{OsstestUpstream} = 'git://xenbits.xen.org/osstest.git master'; + $c{HostProp_DhcpWatchMethod} = 'leases dhcp3 infra:5556'; $c{AuthorizedKeysFiles} = ''; $c{AuthorizedKeysAppend} = '';
ap-common +3 -0
@@ -21,6 +21,8 @@ : ${XENBITS:[email protected]} +: ${TREEBRANCH_OSSTEST_UPSTREAM=`getconfig OsstestUpstream`} + : ${TREE_XEN:=git://xenbits.xen.org/xen.git} : ${PUSH_TREE_XEN:=$XENBITS:/home/xen/git/xen.git}
@@ -76,6 +78,7 @@ fi : ${TREE_LINUX2639:=${KERNEL_SCM}/konrad/xen.git} : ${TAG_LINUX2639:=stable/2.6.39.x} +: ${LOCALREV_OSSTEST:=daily-cron.$branch} : ${LOCALREV_XEN:=daily-cron.$branch} : ${LOCALREV_LINUX:=daily-cron.$branch} : ${LOCALREV_LIBVIRT:=daily-cron.$branch}
ap-fetch-version +18 -1
@@ -90,7 +90,24 @@ ovmf) $TREE_OVMF_UPSTREAM master $LOCALREV_OVMF ;; osstest) - if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then + if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then + OSSTEST_REVISION_MERGE=`repo_tree_rev_fetch_git osstest \ + $TREEBRANCH_OSSTEST_UPSTREAM $LOCALREV_OVMF` + echo >&2 "$TREEBRANCH_OSSTEST_UPSTREAM = $OSSTEST_REVISION_MERGE" + + rm -rf $repos/osstest-merge >&2 + git clone -b pretest $HOME/testing.git $repos/osstest-merge >&2 + + git -C $repos/osstest-merge \ + fetch $repos/osstest $LOCALREV_OVMF:ap-merge >&2 + git -C $repos/osstest-merge merge --no-edit \ + -m "Automerge of $TREEBRANCH_OSSTEST_UPSTREAM into pretest" \ + ap-merge >&2 + + # cr-daily-branch will reset the local tree to this version. + git fetch $repos/osstest-merge pretest + git rev-parse FETCH_HEAD^0 + elif [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then git fetch $HOME/testing.git pretest:ap-fetch >&2 git rev-parse ap-fetch^0 else
ap-push +7 -1
@@ -99,7 +99,13 @@ ovmf) ;; osstest) git push $HOME/testing.git $revision:production - git push $XENBITS:/home/xen/git/osstest.git $revision:master + if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then + # could push to instance specific location, but + # certainly not to master instance's xenbits repo! + : + else + git push $XENBITS:/home/xen/git/osstest.git $revision:master + fi ;; *) echo >&2 "branch $branch ?"
production-config +2 -0
@@ -15,6 +15,8 @@ # You should have received a copy of the GNU Affero General Public License # along with this program. If not, see <http://www.gnu.org/licenses/>. +# This is the master instance, therefore it has no upstream +OsstestUpstream '' JobDB Executive HostDB Executive
Signature · 1 line
2.1.4
Re: [PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances
Ian Campbell writes ("[PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances"):
Quoted 4 lines
We want to arrange that the master XenProject instance continues to test its own pretest branch while any downstream instances will pickup changes from the master instance's production (i.e. tested) branch, which is published at
git://xenbits.xen.org/osstest.git#master.
... ;;
ap-fetch-version ovmf) · line 93
Quoted 3 lines
osstest)
ap-fetch-version +1
-1- if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then + if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then
Shouldn't the OSSTEST_USE_HEAD option override this ? Ie I think this is the wrong way round.
ap-fetch-version ovmf) · line 95
Quoted 2 lines
ap-fetch-version +2
-0+ OSSTEST_REVISION_MERGE=`repo_tree_rev_fetch_git osstest \ + $TREEBRANCH_OSSTEST_UPSTREAM $LOCALREV_OVMF`
OSSTEST_REVISION_MERGE doesn't seem to be an environment variable. I think it should either be honoured if pre-set, or just be an ordinary (lowercase) variable name (in which case it doesn't need the OSSTEST prefix).
ap-fetch-version ovmf) · line 102
Quoted 7 lines
ap-fetch-version +7
-0+ echo >&2 "$TREEBRANCH_OSSTEST_UPSTREAM = $OSSTEST_REVISION_MERGE" + + rm -rf $repos/osstest-merge >&2 + git clone -b pretest $HOME/testing.git $repos/osstest-merge >&2 + + git -C $repos/osstest-merge \ + fetch $repos/osstest $LOCALREV_OVMF:ap-merge >&2
LOCALREV_OVMF (here and earlier). It's not clear to me that you actually need to indirect it; you could just specify a branch name.
The rest looks fine.
Having thought about how to override things, particularly:
ap-push ovmf) · line 106
Quoted 5 lines
ap-push +5
-0+ if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then + # could push to instance specific location, but + # certainly not to master instance's xenbits repo! + : + else
I think ap-fetch-* and ap-push should have a general purpose hook which occurs just after ap-common is loaded, something like this:
diff --git a/ap-fetch-version b/ap-fetch-version
index 33aaf00..a3028f6 100755
--- a/ap-fetch-version
+++ b/ap-fetch-version
@@ -31,6 +31,8 @@ if info_linux_tree "$branch"; then
exit 0
fi
+. ${OSSTEST_HOOK_AP_FETCH-/dev/null}
+
case "$branch" in
xen-3.*)
./sg-hg-heads sh -ec 'We can postpone actually writing that until someone needs it.
Ian.
Re: [PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances
Quoted 5 lines · Ian Jackson
Ian Campbell writes ("[PATCH OSSTEST 2/5] Handle osstest's own local push gate in non-master production instances"):
Quoted 4 lines
We want to arrange that the master XenProject instance continues to test its own pretest branch while any downstream instances will pickup changes from the master instance's production (i.e. tested) branch, which is published at
git://xenbits.xen.org/osstest.git#master.
Quoted 7 lines
... ;;
ap-fetch-version ovmf) · line 93
Quoted 3 lines
osstest)
ap-fetch-version +1
-1- if [ "x$OSSTEST_USE_HEAD" != "xy" ] ; then + if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; thenShouldn't the
OSSTEST_USE_HEADoption override this ? Ie I think this is the wrong way round.
Yes, I think you are right.
I think I just ended up with this because I was testing via the standalone wrapper which sets USE_HEAD, but I'll test another way.
Quoted 7 lines
ap-fetch-version ovmf) · line 95
Quoted 2 lines
ap-fetch-version +2
-0+ OSSTEST_REVISION_MERGE=`repo_tree_rev_fetch_git osstest \ + $TREEBRANCH_OSSTEST_UPSTREAM $LOCALREV_OVMF`
OSSTEST_REVISION_MERGEdoesn't seem to be an environment variable. I think it should either be honoured if pre-set, or just be an ordinary (lowercase) variable name (in which case it doesn't need the OSSTEST prefix).
OK. Any preference as to which?
As it stands the only reason for the variable is to sink the output from repo_tree_rev_fetch and log it.
Quoted 11 lines
ap-fetch-version ovmf) · line 102
Quoted 8 lines
ap-fetch-version +7
-0+ echo >&2 "$TREEBRANCH_OSSTEST_UPSTREAM = $OSSTEST_REVISION_MERGE" + + rm -rf $repos/osstest-merge >&2 + git clone -b pretest $HOME/testing.git $repos/osstest-merge >&2 + + git -C $repos/osstest-merge \ + fetch $repos/osstest $LOCALREV_OVMF:ap-merge >&2
LOCALREV_OVMF(here and earlier). It's not clear to me that you actually need to indirect it; you could just specify a branch name.
OK.
Quoted 27 lines
The rest looks fine.
Having thought about how to override things, particularly:
ap-push ovmf) · line 106
Quoted 5 lines
ap-push +5
-0+ if [ x"$TREEBRANCH_OSSTEST_UPSTREAM" != x ] ; then + # could push to instance specific location, but + # certainly not to master instance's xenbits repo! + : + elseI think ap-fetch-* and ap-push should have a general purpose hook which occurs just after ap-common is loaded, something like this:
diff --git a/ap-fetch-version b/ap-fetch-version index 33aaf00..a3028f6 100755 --- a/ap-fetch-version +++ b/ap-fetch-version @@ -31,6 +31,8 @@ if info_linux_tree "$branch"; then exit 0 fi +. ${OSSTEST_HOOK_AP_FETCH-/dev/null} + case "$branch" in xen-3.*) ./sg-hg-heads sh -ec '
Quoted 1 line
We can postpone actually writing that until someone needs it.
Gladly ;-)
Ian.
PATCH 3/5 cambridge: Add a crontab
In reply to Ian CampbellRight now all it does is keep osstest itself up to date.
--- crontab-cambridge | 5 +++++ 1 file changed, 5 insertions(+) create mode 100644 crontab-cambridge
crontab-cambridge new file +5 -0
@@ -0,0 +1,5 @@ +PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin [email protected],[email protected] +# m h dom mon dow command +49 1 * * * cd testing.git && BRANCHES=osstest ./cr-for-branches branches -w "./cr-daily-branch --real" +3 4 * * * savelog -c28 testing.git/tmp/cr-for-branches.log >/dev/null
Signature · 1 line
2.1.4
Re: [PATCH OSSTEST 3/5] cambridge: Add a crontab
Ian Campbell writes ("[PATCH OSSTEST 3/5] cambridge: Add a crontab"):
Quoted 1 line
Right now all it does is keep osstest itself up to date.
PATCH 4/5 cambridge: Do not try to push harness to XenProject instance output
In reply to Ian CampbellBy arranging for cr-publish-flight-logs to ignore --push-harness if either of HarnessPublishGitRepoDir or HarnessPublishGitUserHost are not specified
---
Perhaps we should have our own tree for such things, but for now just nobble it.
--- cr-publish-flight-logs | 4 ++++ production-config-cambridge | 3 --- 2 files changed, 4 insertions(+), 3 deletions(-)
cr-publish-flight-logs +4 -0
@@ -43,6 +43,10 @@ if ($push_harness) { my $githost= $c{HarnessPublishGitUserHost}; my $gitdir= $c{HarnessPublishGitRepoDir}; + if (!$githost || !$gitdir) { + logm("Nowhere to publish harness to"); + return 0; + } system_checked("git push $githost:$gitdir HEAD:refs/heads/flight-$flight"); system_checked("ssh $githost 'cd $gitdir && git update-server-info'"); }
production-config-cambridge +0 -3
@@ -50,9 +50,6 @@ GlobalLockDir /export/home/osstest/testing.git LogsPublish= "$c{Publish}/logs" ResultsPublish= "$c{Publish}/results" -HarnessPublishGitUserHost [email protected] -HarnessPublishGitRepoDir git/osstest.git - ReportTrailer= <<END Logs, config files, etc. are available at $c{ReportHtmlPubBaseUrl}
Signature · 1 line
2.1.4
Re: [PATCH OSSTEST 4/5] cambridge: Do not try to push harness to XenProject instance output
Ian Campbell writes ("[PATCH OSSTEST 4/5] cambridge: Do not try to push harness to XenProject instance output"):
Quoted 3 lines
By arranging for cr-publish-flight-logs to ignore --push-harness if either of HarnessPublishGitRepoDir or HarnessPublishGitUserHost are not specified
PATCH 5/5 cambridge: Stop publishing logs to chiark
In reply to Ian Campbellhttp://osstest.cam.xci-test.com/~osstest/testlogs already exists and points to the live logs directory, so switch PubBaseUrl to that in the Cambridge config such that email reports etc contain it. This won't be externally accessible but I think that won't matter now that the master production instance is elsewhere.
Arrange that cr-publish-flight-logs doesn't publish the corresponding thing if either LogsPublish or ResultsPublish is not set, and unset them in the Cambridge config.
Likewise arrange that cr-ensure-disk-space doesn't do anything if the configuration variable passed as an option is not set, and unset Publish (the base for {Logs,Results}Publish) in the Cambridge config.
--- cr-ensure-disk-space | 2 ++ cr-publish-flight-logs | 4 ++-- production-config-cambridge | 6 +----- 3 files changed, 5 insertions(+), 7 deletions(-)
cr-ensure-disk-space +2 -0
@@ -40,6 +40,8 @@ die unless @ARGV==1; our ($cfgbase) = @ARGV; +exit 0 unless $cfgbase; + csreadconfig(); open LOCK, "> $c{GlobalLockDir}/publish-lock" or die $!;
cr-publish-flight-logs +2 -2
@@ -61,5 +61,5 @@ sub copydir ($$) { $!=0; $?=0; system @cmd; die "rsync $? $!" if $? or $!; } -copydir("$c{Logs}/$flight/", "$c{LogsPublish}/$flight"); -copydir("$c{Results}/", "$c{ResultsPublish}"); +copydir("$c{Logs}/$flight/", "$c{LogsPublish}/$flight") if $c{LogsPublish}; +copydir("$c{Results}/", "$c{ResultsPublish}") if $c{ResultsPublish};
production-config-cambridge +1 -5
@@ -40,16 +40,12 @@ TestHostKeypairPath /export/home/osstest/.ssh/id_rsa_osstest GitCacheProxy git://drall.uk.xensource.com:9419/ -PubBaseUrl http://www.chiark.greenend.org.uk/~xensrcts +PubBaseUrl http://osstest.cam.xci-test.com/~osstest/testlogs ReportHtmlPubBaseUrl="$c{PubBaseUrl}/logs" ResultsHtmlPubBaseUrl="$c{PubBaseUrl}/results" -Publish [email protected]:/home/ian/work/xc_osstest GlobalLockDir /export/home/osstest/testing.git -LogsPublish= "$c{Publish}/logs" -ResultsPublish= "$c{Publish}/results" - ReportTrailer= <<END Logs, config files, etc. are available at $c{ReportHtmlPubBaseUrl}
Signature · 1 line
2.1.4
Re: [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark
Ian Campbell writes ("[PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark"):
Quoted 26 lines
http://osstest.cam.xci-test.com/~osstest/testlogs already exists and points to the live logs directory, so switch PubBaseUrl to that in the Cambridge config such that email reports etc contain it. This won't be externally accessible but I think that won't matter now that the master production instance is elsewhere.
Arrange that cr-publish-flight-logs doesn't publish the corresponding thing if either LogsPublish or ResultsPublish is not set, and unset them in the Cambridge config.
Likewise arrange that cr-ensure-disk-space doesn't do anything if the configuration variable passed as an option is not set, and unset Publish (the base for {Logs,Results}Publish) in the Cambridge config.
Signed-off-by Ian Campbell <[email protected]>--- cr-ensure-disk-space | 2 ++ cr-publish-flight-logs | 4 ++-- production-config-cambridge | 6 +----- 3 files changed, 5 insertions(+), 7 deletions(-)cr-ensure-disk-space +0
-0
cr-ensure-disk-space die unless @ARGV==1; · line 45
Quoted 6 lines
cr-ensure-disk-space +1
-0@@ -40,6 +40,8 @@ die unless @ARGV==1; our ($cfgbase) = @ARGV; +exit 0 unless $cfgbase; csreadconfig();
Don't you mean "unless $c{$cfgbase}" ? And then it has to come after csreadconfig. (Did you test this ?)
The rest LGTM.
Thanks,
Ian.
Re: [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark
Quoted 28 lines · Ian Jackson
Ian Campbell writes ("[PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark"):
Quoted 27 lines
http://osstest.cam.xci-test.com/~osstest/testlogs already exists and points to the live logs directory, so switch PubBaseUrl to that in the Cambridge config such that email reports etc contain it. This won't be externally accessible but I think that won't matter now that the master production instance is elsewhere.
Arrange that cr-publish-flight-logs doesn't publish the corresponding thing if either LogsPublish or ResultsPublish is not set, and unset them in the Cambridge config.
Likewise arrange that cr-ensure-disk-space doesn't do anything if the configuration variable passed as an option is not set, and unset Publish (the base for {Logs,Results}Publish) in the Cambridge config.
Signed-off-by Ian Campbell <[email protected]>--- cr-ensure-disk-space | 2 ++ cr-publish-flight-logs | 4 ++-- production-config-cambridge | 6 +----- 3 files changed, 5 insertions(+), 7 deletions(-)cr-ensure-disk-space +0
-0@@ -40,6 +40,8 @@ die unless @ARGV==1;
Quoted 7 lines
cr-ensure-disk-space die unless @ARGV==1;
Quoted 5 lines
cr-ensure-disk-space +1
-0our ($cfgbase) = @ARGV; +exit 0 unless $cfgbase; csreadconfig();Don't you mean "unless $c{$cfgbase}" ?
I think I do, yes.
Quoted 1 line
And then it has to come after csreadconfig. (Did you test this ?)
I was running in standalone mode, so I don't think this code will have actually been run, I was relying on my ability to reason about the code, which apparently failed :-/.
Shall I run an adhoc flight with this change in it?
Ian.
Re: [PATCH OSSTEST 5/5] cambridge: Stop publishing logs to chiark
Quoted 1 line · Ian Campbell
Shall I run an adhoc flight with this change in it?
FYI, I think I've now figured out how to run something approximating a "proper" osstest flight from something other than ~osstest/testing.git via the cronjob, so one is in progress now...
Ian.