mg-*: easier transient autoalloc repro
cover mg-*: easier transient autoalloc repro
Ian Jackson (4):
mg-repro-setup: Do all builds in their own tasks, regardless
mg-transient-task: New utility
mg-repro-setup: Introduce `statictask' variable
mg-repro-setup: New mode --autoalloc
mg-repro-setup | 45 +++++++++++++++++++++++++++++++++------------ mg-transient-task | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+), 12 deletions(-) create mode 100755 mg-transient-task
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 1/4 mg-repro-setup: Do all builds in their own tasks, regardless
In case OSSTEST_TASK was set by the caller, unset it. Unsetting it will cause each sg-run-job (inside mg-execute-task) to become its own task.
--- mg-repro-setup | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
mg-repro-setup +2 -1
@@ -247,7 +247,8 @@ progress "new flight is $flight" if [ "x$rebuilds_flight" != x ]; then progress "running build(s) $rebuilds_flight" - ./mg-execute-flight -B$rebuilds_blessing --progress-fd=2 \ + env -u OSSTEST_TASK \ + ./mg-execute-flight -B$rebuilds_blessing --progress-fd=2 \ "${mgexecflags[@]}" $rebuilds_flight mro=tmp/$rebuilds_flight.mro
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 2/4 mg-transient-task: New utility
In reply to Ian JacksonWill be useful in combination with new mode(s) of mg-repro-setup.
--- mg-transient-task | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100755 mg-transient-task
mg-transient-task new file +32 -0
@@ -0,0 +1,32 @@ +#!/usr/bin/tclsh8.5 +# -*- Tcl -*- +# usage: ./mg-transient-task PROGRAM [ARGS...] + +# This is part of "osstest", an automated testing framework for Xen. +# Copyright (C) 2009-2019 Citrix Inc. +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU Affero General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU Affero General Public License for more details. +# +# You should have received a copy of the GNU Affero General Public License +# along with this program. If not, see <http://www.gnu.org/licenses/>. + +package require Tclx + +source ./tcl/daemonlib.tcl + +jobdb::become-task [string range "$argv" 0 50] +exec 2>@ stderr perl -I. -e { + use Osstest; + use Osstest::Executive; + csreadconfig(); + findtask() +} +execl [lindex $argv 0] [lrange $argv 1 end]
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 3/4 mg-repro-setup: Introduce `statictask' variable
In reply to Ian JacksonWe are going to make a mode where we don't set OSSTEST_TASK. The result is that our subprocesses will do whatever they usually do.
Those are mg-allocate (which would allocate for our static task) and mg-execute-flight which will make a dynamic task. We must therefore prevent mg-allocate from running since the allocations would not be useable for the flight execution.
No functional change yet, since nothing sets statictask=false and therefore OSSTEST_TASK would always be set.
--- mg-repro-setup | 28 +++++++++++++++++++--------- 1 file changed, 19 insertions(+), 9 deletions(-)
mg-repro-setup +19 -9
@@ -103,6 +103,7 @@ duration=28d blessing=play skipcapture=true autoalloc=false +statictask=true while true; do case "$1" in
@@ -117,7 +118,7 @@ while true; do -t?*) duration=${arg#-t} ;; --rogue) duration='' ;; --capture) skipcapture=false ;; - --autoalloc-nofree) autoalloc=true ;; + --autoalloc-nofree) autoalloc=true; statictask=true ;; -l*) logfile=${arg#-l} ;; -r^*|-r!*) adjustsets+=("${arg#-r}") ;; # ^ two patterns because there's no way in bash to write
@@ -258,16 +259,25 @@ if [ "x$rebuilds_flight" != x ]; then fi fi -OSSTEST_TASK=$(perl -e ' - use Osstest; - use Osstest::Executive; - csreadconfig(); - findtask(); - printf "%s\n", $ENV{'OSSTEST_TASK'} or die $!; -') -export OSSTEST_TASK +if $statictask; then + OSSTEST_TASK=$(perl -e ' + use Osstest; + use Osstest::Executive; + csreadconfig(); + findtask(); + printf "%s\n", $ENV{'OSSTEST_TASK'} or die $!; + ') + export OSSTEST_TASK +fi if [ "${alloc_idents[*]}" ]; then + if [ x"$OSSTEST_TASK" = x ]; then + # We would have to make a dynamic task and hold the + # fd for it in this script. Would be quite awkward. + echo >&2 'cannot hand-allocate in a way that will free' + exit 1 + fi + progress "allocating ${alloc_idents[*]} ..." alloc_output=tmp/$flight.allocations ./mg-allocate \
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel
PATCH 4/4 mg-repro-setup: New mode --autoalloc
In reply to Ian JacksonThis just involves turning autoalloc on and statictask off. It is most useful with mg-transient-task, as documented.
--- mg-repro-setup | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
mg-repro-setup +12 -2
@@ -28,7 +28,7 @@ usage () { cat <<END HOSTSPEC is [<ident>=][reuse:]<hostname> host must be allocated, skip host-install [<ident>=]wipe:<hostname> host must be allocated, wipe it - [<ident>=]alloc:<resource-spec> \`mg-allocate <resource-spec>', and wipe + [<ident>=]alloc:<resource-spec> \`mg-allocate <resource-spec>', and wipe[*] none: no hosts (should be only HOSTPSEC) (default <ident> is \`host')
@@ -42,6 +42,13 @@ usage () { cat <<END -E... -f... -P as for mg-execute-flight --autoalloc-nofree allocate hosts as for production, but keep them (specify no HOSTSPECS; remember to deallocate later) + --autoalloc use hosts as for production (specify no HOSTSPECS) + and free them afterwards[*] + + [*] If \$OSSTEST_TASK is set on entry, it will be used for these + allocations. So, in particular, + ./mg-transient-task ./mg-repro-setup .... alloc:... + will allocate as specified, but discard the hosts when done. REBUILD is --rebuild [-B<blessing>] [-r...]
@@ -105,6 +112,8 @@ skipcapture=true autoalloc=false statictask=true +if [ "x$OSSTEST_TASK" != x ]; then statictask=false; fi + while true; do case "$1" in -*) ;;
@@ -118,6 +127,7 @@ while true; do -t?*) duration=${arg#-t} ;; --rogue) duration='' ;; --capture) skipcapture=false ;; + --autoalloc) autoalloc=true; statictask=false ;; --autoalloc-nofree) autoalloc=true; statictask=true ;; -l*) logfile=${arg#-l} ;; -r^*|-r!*) adjustsets+=("${arg#-r}") ;;
@@ -298,7 +308,7 @@ if [ "${alloc_idents[*]}" ]; then progressf "\n" if $autoalloc; then echo >&2 \ - 'warning: --autoalloc-nofree and HOSTSPECs are do not work well together' + 'warning: --autoalloc[-nofree] and HOSTSPECs are do not work well together' fi fi
Signature · 7 lines
2.11.0 _______________________________________________ Xen-devel mailing list [email protected] https://lists.xenproject.org/mailman/listinfo/xen-devel